statistics

The Statistics: Affirmance Rates

Affirmance rates can be analogized to batting averages, with a district court's Daubert ruling on any given expert representing one "at bat," and with an appellate affirmance of that ruling representing one "hit." Note that a single opinion may rule on multiple experts.

Formulaically:

Admissibility affirmance rate = X/A, where:

A = the number of experts whose testimony the district courts decided to admit (in decisions subsequently reviewed on appeal); and

X = the number of such rulings that the appellate tribunal affirmed.

Similarly:

Exclusion affirmance rate = Y/E, where:

E = the number of experts whose testimony the district courts excluded (in decisions later reviewed on appeal); and

Y = the number of such exclusions that the appellate tribunal upheld.

Each circuit's affirmance rates are given at the top of the page for that circuit, for a sample comprising the decisions summarized on that page. Rates for the combined circuits appear on the main Circuits page.

Caveats and Explications

Obviously, these statistics are just one highly vague indicator of how things happen to be going, and a false level of precision should not be ascribed to them. For one thing, the sample used here consists only of (recent) appellate decisions, which makes for a relatively small sample size, especially at the individual circuit level. For another, the mere circumstance that data can be organized and quantified does not guarantee that the results portray anything real.

The Rules

Reasonable lawyers could differ on how any given decision should be counted in these statistics, or even on whether it should be counted at all. No pretense is made that the counting rules employed here are the only defensible ones, or that their application in particular cases is uncontroversially uniform. There is probably an irreducible element of arbitrariness in the project. Nevertheless, the intellectual consumer is entitled to a description of the counting rules adopted.

Rule #1: Primary attention is given to how the court of appeals perceives the issues. If it seems likely that the appellate tribunal itself would characterize its ruling as springing from Daubert's evidentiary teachings, then the appellate decision generally gets counted in the sample. If the court of appeals appears to perceive its disposition as resting on other grounds, the decision does not get counted.

Rule #2: How an appellate decision gets counted depends on its treatment of the lower court's Daubert ruling, and not on whether the district court judgment is affirmed. Thus in a case where the jury's verdict for plaintiff is upheld because the exclusion of defendant's expert was harmless error, the decision gets reckoned as a reversal of the district court's Daubert ruling, not as an affirmance of the verdict.

Rule #3: As in baseball, there's no such thing as half a hit. Mixed decisions, where the trial court's rulings on some expert are affirmed in part and reversed in part, get chalked up as one entire affirmance or one entire reversal. Factors influencing this choice include the relative importance of the affirmed points versus the reversed points, the effect of the Daubert decision on the case's ultimate appellate disposition, and various other ineffable considerations.

Rule #4: One counts expert by expert, not case by case. If the district court excludes two experts and the court of appeals affirms as to one and reverses as to one, the result is counted as one reversal and one affirmance.

Rule #5: To avoid distortions introduced by runaway scores, Rule #4 is subject to a bulk-treatment exception. Suppose, for instance, that a plaintiff offers affidavits from five chiropractors for the proposition that plaintiff's neurological damage was caused by an auto accident. If the court of appeals affirms the exclusion of the chiropractors' testimony on grounds generally applicable to all (e.g., the dubious reliability of chiropractic as a diagnostic methodology for neurological disorders), and especially if it does so without a separate discussion of each chiropractor, then we have one affirmance, not five.

Rule #6: All rules are subject to exception under an overarching rule of reason.

Further Research

If readers find these counting methods inappropriate, they are strongly encouraged to do computations under different rules of their own.