Forensic evidence largely not supported by sound science – now what?

The Conversation

Forensic science has become a mainstay of many a TV drama, and it’s just as important in real-life criminal trials. Drawing on biology, chemistry, genetics, medicine and psychology, forensic evidence helps answer questions in the legal system. Often, forensics provides the “smoking gun” that links a perpetrator to the crime and ultimately puts the bad guy in jail.

Shows like “CSI,” “Forensic Files” and “NCIS” cause viewers to be more accepting of forensic evidence. As it’s risen to ubiquitous celebrity status, forensic science has become shrouded in a cloak of infallibility and certainty in the public’s imagination. It seems to provide definitive answers. Forensics feels scientific and impartial as a courtroom weighs a defendant’s possible guilt – looking for proof beyond a reasonable doubt.  

But the faith the public and the criminal justice system place in forensic science far outpaces the amount of trust it deserves.

For decades, there have been concerns about how the legal system uses forensic science. A groundbreaking 2009 report from the National Academy of Sciences finally drew the curtain back to reveal that the wizardry of forensics was more art than science. The report assessed forensic science’s methods and developed recommendations to increase validity and reliability among many of its disciplines.

These became the catalyst that finally forced the federal government to devote serious resources and dollars to an effort to more firmly ground forensic disciplines in science. After that, governmental agencies, forensic science committees and even the Department of Defense responded to the call. Research to this end now receives approximately US$13.4 million per year, but the money may not be enough to prevent bad science from finding its way into courtrooms.

This fall, the President’s Council of Advisors on Science and Technology (PCAST) released its own report on forensic science. It’s a more pronounced acknowledgment that the discipline has serious problems that require urgent attention. Some scientific and legal groups are outraged by or doubtful of its conclusions; others have praised them.

As someone who has taught forensic evidence for a decade and dedicated my legal career to working on cases involving forensic science (both good and bad), I read the report as a call to address foundational issues within forensic disciplines and add oversight to the way forensic science is ultimately employed by the end user: the criminal justice system.

Is any forensic science valid?

The President’s Council of Advisors on Science and Technology recognized ongoing efforts to improve forensic science in the wake of the 2009 NAS report. Those efforts focused on policy, best practices and research around forensic science, but, as with any huge undertaking, there were gaps. As PCAST noted, forensic science has a validity problem that is in desperate need of attention.

PCAST focused on what’s colloquially termed “pattern identification evidence” – it requires an examiner to visually compare a crime scene sample to a known sample. PCAST’s big question: Are DNA analysis, bite marks, latent fingerprints, firearms identification and footwear analysis supported by reproducible research, and thus, reliable evidence?

They were looking for two types of validity. According to PCAST, foundational validity means the forensic discipline is based on research and studies that are “repeatable, reproducible, and accurate,” and therefore reliable. The next step is applied validity, meaning the method is “reliably applied in practice.” In other words, for a forensic discipline to produce valid evidence for use in court, there must be (1) reproducible studies on its accuracy and (2) a method used by examiners that is reproducible and accurate.

Among the forensic science they assessed, PCAST found single-sourced DNA analysis to be the only discipline that was valid, both foundationally and as applied. They found DNA mixture evidence – when DNA from more than one person is in a sample, for instance from the victim and the perpetrator, multiple perpetrators or due to contamination – to be only foundationally valid. Same with fingerprint analysis.

Firearms identification had just the potential for foundational validity, but the research that could support it hasn’t been done yet. Footwear analysis lacked studies even showing potential for foundational validity. And bite mark analysis has a low chance of achieving any validity; the PCAST report advised “against devoting significant resources” to it.

All these types of evidence are widely used in thousands of trials each year. Many additional cases never even go to trial because this supposedly definitive evidence seems damning and compels defendants to plead guilty. But the lack of reliable science supporting these disciplines undermines the evidence which, in turn, undermines criminal convictions.

Risks of lacking validity

When forensic methods are not validated but nevertheless perceived as reliable, wrongful convictions happen.

For example, the field of forensic odontology presumes that everyone has a unique bite mark. But there’s no scientific basis for this assumption. A 2010 study of bite marks from known biters showed that skin deformations distort bite marks so severely that current methods of analysis could not accurately include or exclude a person based on the pattern left by their teeth.

In 1986, Bennie Starks was convicted of rape and other crimes after forensic odontology experts testified he was the source of a bite mark on the victim. In 2006, DNA test results showed Starks could not have been the perpetrator. Starks spent 20 years in prison for a crime he did not commit because of faulty evidence from an unreliable discipline. More recently, the Texas Forensic Science Commission recommended a flat-out ban on bite mark evidence.

Like in Starks’ case, questionable forensic evidence plays a significant role in at least half of overturned convictions, according to the Innocence Project. Once a verdict comes in, it becomes a Sisyphean task to undo it – even if newly discovered evidence undermines the original conviction. It’s next to impossible for people once convicted to get their cases reconsidered.

At the moment, only two states (Texas and California) permit a defendant to appeal a conviction if the scientific evidence or the expert who testified is later discredited. More laws like these are needed, but it’s politically a hard sell to grant more rights and avenues of appeal to convicts. So even if the science is undermined or completely discredited, a prisoner is often at the mercy of a court as it decides whether to grant or deny an appeal.

What should be admissible?

The PCAST report recommended judges consider both the foundational and applied validity of the forensic discipline that produced any evidence before admitting expert testimony. This includes ensuring experts testify to the limitations of the analysis and evidence. For example, the justice system traditionally considers fingerprint evidence as an “identification” – for instance, the thumbprint recovered from the crime scene was made by the defendant’s thumb. No one ever testifies that there are little scientific data establishing that fingerprints are unique to individuals. The same holds true for other types of pattern identification evidence such as firearms, toolmarks and tire treads.

The National District Attorneys Association (NDAA) was critical of the PCAST report. It countered that there actually is scientific data validating these forensic fields, but members of PCAST did not adequately consult subject-matter experts. The NDAA also worried that if courts required stronger scientific validity before allowing evidence into court, it would hamstring the entire investigative process.

The NDAA concluded that judges should continue to be the ones who decide what makes evidence reliable and thus admissible. It asserted that the stringent requirements to become expert witnesses, along with the ability to cross-examine them in court, are enough to guarantee reliable and admissible evidence.

But should the admissibility of scientific processes – which ought to be grounded in their proven ability to produce reliable evidence – be determined by people who lack scientific backgrounds? I would argue no.

Pattern identification evidence shouldn’t be excluded from cases wholesale, but forensic evidence needs to be placed into context. When the human eye is the primary instrument of analysis, the court, the attorneys and the jury should be fully aware that certainty is unattainable, human error is possible, and subjectivity is inherent.

Reliance upon the adversary system to prevent wrongful convictions and weed out junk science requires a leap of faith that ultimately undermines the integrity of the criminal justice system. Counting on cross-examination as an effective substitute for scientific rigor and research can’t be the answer (although it has been for more than a century).

The PCAST report is yet another wake-up call for the criminal justice system to correct the shortcomings of forensic science. We demand that guilt be proven beyond a reasonable doubt; we should also demand accurate and reliable forensics. Without improvement, we can’t trust forensic science to promote justice.

https://theconversation.com/forensic-evidence-largely-not-supported-by-sound-science-now-what-67413