legal cases that illustrate the role of weight (as well as higher order probability)

[marcellodibello]

Some of the cases below serve to illustrate possible applications of the concept of weight but also applications of higher order probability

Looks like the main difference is this:

• in case of higher order probability, we assess the overall evidence available and its higher order uncertainty

• in the case of weight, the question is how further evidence could change the strength of the evidence available

So the question of weight seems connected with the question of resilience and stability. Is this correct @rfl-urbaniak?

[marcellodibello]

Whole food overpricing

Not sure if this is immediately relevant for weight, but it is an interesting case.

District court decision In re Whole Foods Mkt. Grp., Inc. Overcharging Litig., 167 F. Supp. 3d 524 (S.D.N.Y. 2016) https://casetext.com/case/in-re-whole-foods-mkt-grp-inc-overcharging-litig

FACTS and EVIDENCE: Plaintiffs allege that WF overcharged them for some prepackaged products they purchased during 2014-15. They base their claim not on individualized evidence, but rather, a statistical sampling analysis done during the 2014-15 period by NY state. The study found that 89% of the sampled products, across 80 types of products, to be overpriced. (More specifically, NY Department of Commerce and Agriculture DCA press release stated: "DCA tested packages of 80 different types of pre-packaged products and found all of the products had packages with mislabeled weights. Additionally, 89 percent of the packages tested did not meet the federal standard for the maximum amount that an individual package can deviate from the actual weight, which is set by the U.S. Department of Commerce. The overcharges ranged from $0.80 for a package of pecan panko to $14.84 for a package of coconut shrimp.")

(Incidentally, note that on December 23, 2015, Whole Foods and the DCA agreed on a 500k settlement in exchange of DCA halting its investigation against WF).

District court dismissed the allegation by the plaintiffs and held they lacked Article 3 standing for two reasons.

One: unclear sampling methodology. ("How many total packages of each identified product did the DCA test, and was that number statistically significant so as to support an inference that the weighting practices found by DCA applied to other packages of the same product sold by Whole Foods? From how many stores did the DCA purchase each of the identified products, and were its findings as to the incidence of over-weighting consistent across these stores? Did the DCA purchase the specific types of products allegedly bought by John and Bassolino from the stores in which they shopped, and did the products bought from those stores bear inflated weights?")

Two: lack of particularized evidence that the the plaintiff did purchase products that were overpriced. ("even if one assumes arguendo that the DCA's testing methodology was sturdy enough to give rise to statistically significant conclusions as to the subset of products bought by John and Bassolino at the stores and during the time periods in which they bought them, there is no non-speculative basis on which to conclude that the particular packages of Whole Food products John and Bassolino bought were overweighted.")

Appeal 2nd Circuit John v. Whole Foods Mkt. Grp., Inc., 858 F.3d 732 (2d Cir. 2017) https://casetext.com/case/john-v-whole-foods-mkt-grp-inc-1

Second Circuit disagreed with the lower court and remanded the decision to lower court again for further consideration. ("we conclude that John has plausibly alleged that he suffered an injury in fact by pleading both the frequency of his purchases and the systematic overcharging of pre-packaged foods at the Whole Foods stores he patronized. CONCLUSION. We VACATE the District Court's judgment and REMAND for further proceedings consistent with this opinion.")

District court SECOND decision In re Whole Foods Mkt. Grp., Inc. Overcharging Litig., 397 F. Supp. 3d 406 (S.D.N.Y. 2019) https://casetext.com/case/in-re-whole-foods-mkt-grp-inc-overcharging-litig-1

This a very long and complicated decision, with may twists and turns, but it essentially dismisses the allegation by plaintiffs for lack of standing. The court looked more closely at the DCA sampling analysis, and both parties submitted more evidence about the case. The court leveled a key criticism to the study---that it was methodologically flawed and unrepresentative, and thus it did not even show that 89% of the products were overpriced. ("During the audit test, the DCA inspector selected products from a lot and manually weighed them. Only if the products tested in the audit phase were found to have a material discrepancy were they subjected to the second phase, consisting of statistical analysis testing. If the products tested in the audit phase were not found to have a material weight discrepancy, they were returned to the shelf. Their weights—and the number of fully compliant packages—went unrecorded. Under these circumstances, whatever the wisdom of this methodology, the outcome of the second phase of the DCA investigation cannot be taken to reflect, at all, the incidence of short-weighted food products at Whole Foods. That phase by definition cherrypicked only lots that had failed the initial phase and excluded those that had passed. This methodology was by definition and design unrepresentative.")

weight of evidence question in this case

• If the statistical evidence against WF was based on better data and analyzed through a more adequate model, how would that have changed the existing evidence?

higher order probability question in this case

• based on the given statistics and model, one might conclude that it is 89% likely that the products the customer purchased were overpriced. what is the higher order uncertainty that applies to the 89% probability assessment?

[marcellodibello]

Interesting cases on using data and competing statistical models for supporting claims of tax evasion and tax fraud.

Sample statistics as convincing evidence: A tax fraud case by Jostein Lillestøl https://openaccess.nhh.no/nhh-xmlui/bitstream/handle/11250/2564537/1218.pdf?sequence=1

the question here is how this case, mostly based on statistical evidence, should be analyzed using probabilities, posterior and prior and higher order probabilities

[marcellodibello]

Interesting discussion on using data and statistics (risk ratios) to establish causality on tort cases

From Statistical Evidence to Evidence of Causality by A. P. Dawid, M. Musio, and S. E. Fienberg https://arxiv.org/pdf/1311.7513

See also section 4.6 Issues Involving Epidemiologic Evidence in Tort Law of this paper The Role of Statistical Evidence in Civil Cases by Joseph L. Gastwirth

https://www.annualreviews.org/content/journals/10.1146/annurev-statistics-031219-041238#right-ref-B33

[marcellodibello]

Tin Box Case

Swedish case

p. 140 of Dahlman's article Information economics in the criminal standard of proof https://academic.oup.com/lpr/article/21/3-4/137/7077281

Man confesses to having killed an elderly women while attempting to steal money in a tin box in her house.

The incriminating evidence consists of:

• confession
• defendant's knife with blood matching victim
• defendant detailed story of what happened

At trial D's lawyer suggests that D is trying to cover up for one of his sons, both with no criminal record. D insists he did it.

Exculpatory evidence:

• angle of stabbing is odd since D is short
• D's sons are taller

One item of evidence is missing:

• the tin box in which the elderly woman's money was stored was not examined for matching fingerprints by police. They thought it was unnecessary since D confessed. Box was later wiped cleaned and no testing was possible.

D was acquitted because evidence is not robust enough.

Weight question in this case:

• Had evidence about fingerprints on the tin box been available at trial, how would the overall evidence have changed?

[marcellodibello]

just to clarify, the question of weight is something like this:

• If we know E is missing, we know roughly what it is but we do not know its value (say incriminating or exculpatory), how would we expect the overall strength of the evidence after adding E to the other evidence?

So weight is the (expected ?) difference (or ratio) between prior higher order distribution (based on other evidence without E) and posterior higher order distribution (based on all evidence including E, accounting for all possible values of E)

[marcellodibello]

The Case of the Missing Fingers

p. 141 of Dahlman's article Information economics in the criminal standard of proof https://academic.oup.com/lpr/article/21/3-4/137/7077281

Swedish case

A video shows a men part of ISIS beheading another person. The question is identity. The man in the video has a pair of missing fingers. A suspect is identified, has missing fingers and matches other characteristics (travelled to Syria, participated in ISIS etc.). Suspect denies beheading the person in the video.

Key identification evidence:

• suspect's missing fingers are missing in the same places as the missing fingers of the person in the video.
• probability of random match in Swedish population based on police database is assessed to be 2/20,000
• only 2 matches in 20,000 entry database
• matching fingers of people in database are of people who died at the time of the crime

Missing evidence:

• database is based on Swedish population, not ISIS members, random match probability could have been different

Suspect is acquitted or charges because evidence was not robust

Interesting quotation: "The court explains that a random match probability of 1 in 10 000 would have been sufficient for proof beyond reasonable doubt, given the other circumstances of the case, if this probability had been robust, but in the absence of more reference data on people affiliated with ISIS it is not sufficiently robust for the standard of proof in criminal cases." (p. 142)

Weight question in this case:

• Had an ISIS database been available at trial, how would the overall evidence have changed?

[marcellodibello]

Salem case

Aggravated murder case.

Victim was stabbed to death in her house. Defendant is convicted. He appeals. Oregon App Ct first grants reconsideration by post-conviction court. Then, post-conviction court rejects defendant’s arguments. Defendant appeals again. Oregon App Ct disagrees with post- conviction court and agrees with defendant.

See Jesse Lee Johnson v. Jeff Premo 2021 Oregon Appellate case. Link to decision: https://law.justia.com/cases/oregon/court-of-appeals/2021/a159635.html

Evidence against defendant:

• fingerprint match in victim’s home
• cigarette butt in victim’s home genetic match
• statement by informant that defendant said “offed the bitch to rob her” • jewelry found with defendant matched victim’s
• blood boot prints in victim’s house match defedant’s boot prints

Exculpating evidence:

• negative genetic match between weapon and defendant
• negative genetic match between blood in victim’s home and defendant
• defendant’s boots did not test positive for blood

Missing evidence (not presented at trial, fault of counsel and police):

• testimony by women living across the street. She saw white man enter victim’s house first, then loud noises and screams followed and finally he run away. Later she saw black man enter the house and then leave. Defendant is black.

Trial:

• Defendant is convicted

Post conviction court:

• counsel was at fault for not presenting neighbor testimony
• still, no prejudice occurred since the testimony corroborates other evidence which was presented

Appellate court:

• counsel was at fault, agrees with post-conviction coirt on this point
• but, contra post-conviction court, prejudice did occur since the testimony of the neighbor can be the basis of a solid defense case (=white man killed woman before defendant came in: black man came in afterwards)

weight of evidence in this case

• had the witness testimony been introduced, how would the overall evidence have changed? Is the post conviction court right that not much would have changed since the evidence would still be strongly incriminating or is the appellate court right?

*challenge for us This seems a simple case yet non-trivial, and we could use it as a proof of concept to showcase our formal modeling:

• aggregating various pieces of evidence
• incriminating and exculpatory evidence
• quantitative evidence (genetic match) and qualitative (eyewitness testimonies)
• causal stories and narratives
• missing evidence and weight What do you think @rfl-urbaniak?

[rfl-urbaniak]

just to clarify, the question of weight is something like this:

* If we know E is missing, we know roughly what it is but we do not know its value (say incriminating or exculpatory), how would we expect the overall strength of the evidence after adding E to the other evidence?

So weight is the (expected ?) difference (or ratio) between prior higher order distribution (based on other evidence without E) and posterior higher order distribution (based on all evidence including E, accounting for all possible values of E)

not exactly, you need to look at a measure of informativeness; I hope the previous discussion clarified this

[rfl-urbaniak]

weight & cases.pdf

[marcellodibello]

discuss spanish cases of false conviction, dna match versus eyewitness testimony, statistics about relative weight of the two types of evidence

CASE 1 -- CASALEIR

"In June 2016, a man with a hoodie and a balaclava invaded the home of an elderly man in A Guarda county in Pontevedra, Spain. He immobilized the elderly resident of the house with isolation tape and robbed him of €50 before leaving. No fingerprints were found on the tape used to restrain the elderly man, but police investigators sent the balaclava, which was left at the scene, to the forensic examination to test for DNA. In the meantime, an eyewitness came forward claiming they had seen the robber. At the police station, the witness identified Jorge Casaleiro as the perpetrator from a series of photographs of possible suspects. Casaleiro was arrested and charged, and his case was brought before a judge before the results from the DNA analyses were in. In 2017, Casaleiro was found guilty of robbery and sentenced to four years and three months in prison. The witness’ testimony and positive identification of him made up the only evidence against him. The results of the forensic analyses, however, later revealed that the DNA found on the balaclava did not match to Casaleiro but to a known criminal whose DNA was already in the police database. In 2021, Casaleiro was exonerated, having spent three years in prison where he also experienced multiple psychotic seizures." (https://www.registryofexonerations.eu/case_details/jorge-casaleiro-robberyburglary-2017/)

• Appellate judgment: "El Tribunal Constitucional ha considerado prueba suficiente para enervar la presunción de inocencia el reconocimiento efectuado en el juicio oral, sin ningún género de dudas, por parte del testigo, a pesar de las irregularidades de los reconocimientos fotográficos" ("The Constitutional Tribunal has considered sufficient caution to substantiate the presumption of innocence of the recognition made in the oral judiciary, without any doubts, by part of the testimony, considering the irregularities of the photographic recognitions")

[marcellodibello]

QUESTION: add example that is not clear cut, along the lines of dog fur evidence, should you do additional testing if you have already matching evidence?

• Contrast two cases: (i) match evidence say (1/10,000 RMP but small sample). What is the weight of a second match (could be positive or negative) that has a RMP again of 1/10,000 but large sample? (ii) Same as (i) but first match has a smaller sample. Might be enough to illustrate how weight vary in light of these circumstances.

• Problem here is, how do we assess the need of doing "additional testing"? Weight does not give us a decision rule, and it only tells us how the higher order distribution will change in light of the additional evidence, right?

[marcellodibello]

Consider imaginary case: Available evidence include eyewitness identification . Question is about weight of DNA evidence. We could imagine:

• key incriminating evidence against defendant is eyewitness identification

• tested it via cross-examination, witness is very confident, distance is close, good lighting, pristine conditions of lineup identification

• we can quantify the strength of eyewitness identification using all the information available

• "under more pristine testing conditions (e.g., initial, uncontaminated memory tests using fair lineups, with no lineup administrator influence, and with an immediate confidence statement). Under those conditions, mock-crime studies and police department field studies have consistently shown that, for adults, (a) confidence and accuracy are strongly related and (b) high-confidence suspect identifications are remarkably accurate."

"Averaged across the three experiments, accuracy associated with the lowest level of confidence was 61.4% correct (this score would be slightly lower had all three studies involved a 50% target-present base rate), whereas accuracy associated with the highest level of confidence was 97.0% correct. "

(The Relationship Between Eyewitness Confidence and Identification Accuracy: A New Synthesis by John T. Wixted and Gary L. Wells, https://journals.sagepub.com/doi/epub/10.1177/1529100616686966)

• (optional) there is other corroborating information (say, defendant has a fresh glass cut on right arm and witness/victim story says that witness broke into victim's car, broke window and robbed her)

• question: there is DNA testing on the way, but it is delayed. Should the judgment be delayed? What is the "weight" of the DNA test result in light of the strong eyewitnesses identification and corroborating information?

[marcellodibello]

Other possible case. Sexual battery. There is testimony of victim that defendant committed abuse, sexual violence etc. Medical examination however does not produce any conclusive evidence of physical abuse. Prosecutor introduces evidence of priors bad acts and other profile evidence to corroborate victim's testimony. What is the weight of this additional evidence? See case: McLean v. State, 934 So. 2d 1248 - Fla: Supreme Court 2006 (https://casetext.com/case/mclean-v-state-16)

Sharon Childress, an advanced registered nurse practitioner, conducted a physical examination of J.N. The exam did not reveal physical evidence of sexual abuse, but Childress testified that this is expected in a case of anal-digital contact unless there has been trauma in addition to the penetration with the finger.

To corroborate J.N.'s testimony, the State sought to introduce evidence of McLean's prior sexual molestation of another boy, whose last name was Chambers.

the trial court instructed the jury that the collateral crime evidence could be considered only for the "purpose of proving opportunity, intent, the absence of mistake or accident on the part of the defendant, or to corroborate the testimony of [J.N.]." The trial court repeated this instruction during the final jury charge.

Accordingly, we held that in cases where the defendant is accused of a sexual battery committed in the familial setting, evidence of a prior sexual battery committed within the familial setting is admissible under the Williams rule because this evidence is relevant to corroborate the victim's testimony.

[marcellodibello]

Two things for next week: (1) construct an informal model of eyewitness and dna evidence cases with statistics and numbers. (2) cases involving profiling statistics with actual numbers and citation from literature.

[marcellodibello]

EYEWITNESS ID RESEARCH SUMMARY

(The Relationship Between Eyewitness Confidence and Identification Accuracy: A New Synthesis by John T. Wixted and Gary L. Wells, https://journals.sagepub.com/doi/epub/10.1177/1529100616686966)

In eyewitness ID research about confidence/accuracy relationship uses calibration curves (see paper above):

Their question is: Given that an eyewitness has a particular level of confidence in his or her ID of a suspect, how accurate is that ID likely to be? With regard to that question, a calibration curve provides much more relevant information than a correlation coefficient. Once this fact was understood, ... researchers began to measure the confidence-accuracy relationship by plotting calibration curves." (p. 23)

PRISTINE CONDITIONS (p. 20):

1. Include only one suspect per lineup

2. The suspect should not stand out in the lineup

3. Caution that the offender might not be in the lineup

4. Use double-blind testing

5. Collect a confidence statement at the time of the identification

How the CALIBRATION formula (for choosers = cases in which eyewitness makes an ID, correct or not):

• NUMERATOR: number of (correct) suspect IDs from target-present (TP) lineups made with confidence level c (nSIDTP-c)

• DENOMINATOR: total number of IDs (to suspects SID and fillers alike FI ) made with confidence level c, for target present (TP) and target abesnt (TA) lineups

• $\frac{nSID{TP-c}}{nSID{TP-c}+nFID{TP-c}+nFI{TA-c}}$

the results shown in Figure 2a are fairly typical of calibration studies, and they show that low-confidence IDs (c = 0%–20%) are associated with low accuracy (26.6% correct), whereas high-confidence IDs (c = 90%–100%) are associated with much higher accuracy (84.9% correct). (p. 23)

IMPORTANT CAVEAT (base rates of TP and TA lineups):

These results ... correspond to the 50% base rate of target-present (TP) lineups used in that study. As we discuss in more detail later, real police lineups may contain a guilty suspect less than 50% of the time. In such cases, the accuracy rates for choosers would be correspondingly lower than the values shown in Figure 2a. (p. 23)

the forensically relevant question is this: Given that the eyewitness picked the suspect with a particular level of confidence, how likely is it that the suspect is guilty? The answer to that question is provided by a CAC plot (p. 23)

We then use representative calibration data and reanalyze those results using the more detailed Bayesian analysis described by Wells et al. (2015). This analysis shows suspect-ID accuracy across the full range of base rates of target-present lineups (instead of limiting the analysis to the 50% base rate typically used in studies, as CAC analysis does) (p. 24)

Formula for ID ACCURACY (p. 25):

• nSID{TP-c} / (nSID{TP-c} + nSID_{TA-c}).

How, then, does one compute the number of innocent-suspect IDs? Using one reasonable approach, the innocent suspect in a target-absent lineup is simply a designated filler, usually the filler that was used to replace the perpetrator’s photo (as in Fig. 1). (p. 25)

• TABLES in Fig 3 p. 27

For the two experiments from Read et al. (1992) shown in Figure 3a and 3b, the base rate of target-present lineups was approximately 50%. Thus, random chance suspect-ID accuracy in these two studies was 50% correct (and, of course, perfect accuracy is 100% correct). For the experiment from Read et al. (1990) shown in Figure 3c, the base rate of target-present lineups (and, therefore, chance accuracy) was approximately 67%. (p. 28)

POLICE FIELD STUDIES (p. 40):

The advantage of a mock-crime study such as the ones considered above is that the experimenter knows if a suspect ID is correct or incorrect, thereby allowing a direct computation of suspect-ID accuracy. In a police department field study, by contrast, it is not known if a suspect ID is correct or incorrect

BASE RATE PROBLME (p. 41):

An issue in generalizing from the lab to the real world is that the base rate of target-present lineups is unknown, and it is quite likely that the base rate will vary from one police department to another, or even from one detective to another, as a function of how much evidence an investigator requires before placing a possible suspect in a lineup

Consider the prior-by-posterior curves that we created for the Wetmore et al. (2015) data as displayed in Figure8. We used Bayes’s theorem to calculate each point in these curves (p. 60)

for moderate-confidence witnesses in the Wetmore et al. data, moving from a 30% base rate to an 80% base rate changed the probability that the suspect was the perpetrator from 74.5% to 96.5% (p. 60)

for high-confidence witnesses, moving from the 30% base rate to the 80% base rate changed the probability that the suspect was the perpetrator from 87.7% to 98.5%, a change of less than 11 percentage points. (p. 61)

And for low-confidence witnesses, moving from the 30% base rate to the 80% base rate changed the probability that the suspect was the perpetrator from 62.8% to 94.0%, a change of over 30 percentage points. (p. 61)

• moderate confidence - bayes factor: from 74.5/30=2.5 to 96.5/80=1.2
• high confidence - bayes factor: from 87.7/30=2.9 to 94/80=1.175
• low confidence - bayes factor: from 62.8/30=2 to 94/30=3

BASE RATE ESTIMATE - 35% (p. 43):

However, the signal-detection model used by Wixted et al. (2016) provided a principled estimate of the base rate in the Houston Police Department. The base-rate estimate that Wixted et al. reported—35%—is just that, an estimate, so it could be wrong. However, it is a principled estimate because it is based on a theory that has long guided thinking about recognition memory in other contexts.

[marcellodibello]

Model of eyewitness ID PLUS match evidence (tests results not yet known, e.g. genetic evidence or other match evidence). So the question is whether we should wait for match evidence test. Strategy: makes strongest possible case for moving on with just eyewitness ID and see what happens.

• Eyewitness ID identification done under pristine conditions, with base rate of 35% (see estimated above).

• Bayes net will look like this: Guilty/innocent -- > eyewitness ID

• Next ask about value/weight of match evidence CASE 1: match/nom-match immediately indicative of guilt or innocence (match <-- guilt/innocent -- > eyewitness ID) CASE 2: match-non-match indirect evidence of guilt/innocence (match <-- presence <-- guilt/innocence -- > eyewitness ID)

• try out various forms of match evidence (saying using the two forms of match evidence in dog fur evidence)

[marcellodibello]

Case using profiling evidence (from my own paper: https://www.journals.uchicago.edu/doi/full/10.1086/705764)

In 1992 a package containing raw opium was delivered to an apartment rented by Neng Vue and Lee Vue, two brothers of Hmong ancestry who lived in the city of Minneapolis. The police monitored the delivery, and the brothers were arrested and brought to trial on opium trafficking charges. To bolster the case against them, the prosecution called an expert witness to the stand who testified that 95 percent of the opium smuggling cases in the Minneapolis area involved people of Hmong ancestry. When paired with the fact that the Hmong compose only 6 percent of the population in the area, the 95 percent estimate yields the following correlation:

Ethnicity: In the Minneapolis area, someone who is of Hmong ancestry is 297 times more likely to be trafficking drugs as compared to someone who is not of Hmong ancestry.2

FOOTNOTE: "If the Hmong commit 95 percent of the opium smuggling crimes and are 6 percent of the population in the area, the remaining 94 percent of the population must commit only 5 percent of such crimes. That is, (.95/.06)/(.05/.94) = 297."

BAYES THEOREM CALCULATIONS:

• Priors: 50% (they were found with lots of drugs in their house)
• P(Hmong | guilty smuggling drugs) = 95%
• P(Hmong | non guilty of smuggling drugs) = 6%
• LR = .95/.06=15.8
• Prior ratio = Pr(guilty of smuggling | found with package)/Pr(not guilty of smuggling | found with package) = 1
• Posterior ratio = 15.8
• Posterior probability is 94% (=15.8/(15.8+1))

???Does this make sense???

Could we say that even though the posterior probability of guilt is very high (unless calculations above are wrong!), the weight of the profiling evidence is very low?