RESEARCH

The evidence record.

Every statistic on this site, its source, and what it cannot prove.

None of the research below is a measurement of Absentia. It is the published record of how discovery is graded and what the empirical literature can and cannot establish.

01 THE RECORD

What can be said.

  1. 01
    FINDING

    73%

    WHAT IT SAYS

    of production cost went to review in RAND's study of 57 large-volume productions. Processing was 19% and collection 8%.

    SOURCE Nicholas M. Pace & Laura Zakaras, RAND, Where the Money Goes (2012)⁠
    LIMIT

    A cost distribution across 57 productions at a diverse set of very large companies. Not a measure of whether any production was complete.

  2. 02
    FINDING

    209 of 131,992

    WHAT IT SAYS

    federal civil cases sampled from filings in 2007-2008 across 19 districts had a spoliation-related motion. That is 0.15%.

    SOURCE Emery G. Lee III, Federal Judicial Center, Motions for Sanctions Based Upon Spoliation of Evidence in Civil Cases (2011)⁠
    LIMIT

    The FJC cautioned that spoliation is likely raised in many cases without becoming a motion. Court disputes are the visible cases, not a failure rate for all productions.

  3. 03
    FINDING

    53%

    WHAT IT SAYS

    of those 209 cases involved electronically stored information.

    SOURCE Federal Judicial Center (2011)⁠
    LIMIT

    A share of the cases found, not of productions made.

  4. 04
    FINDING

    34%

    WHAT IT SAYS

    of ESI spoliation sanctions motions that reached an order were granted: 20 grants against 38 denials.

    SOURCE Federal Judicial Center (2011)⁠
    LIMIT

    A rate within litigated motions that reached an order. It is not a rate of misconduct, and not a rate of missing documents. The report puts the denominator at 65 in one sentence and at 58 in the next; 20 granted and 38 denied is 58, and 58 is what 34% follows from.

  5. 05
    FINDING

    230 of 401

    WHAT IT SAYS

    cases involving motions for sanctions over electronically stored information ended in a sanction award. The survey covers written opinions from cases in federal courts before 1 January 2010.

    SOURCE Dan H. Willoughby Jr., Rose Hunter Jones & Gregory R. Antine, Sanctions for E-Discovery Violations: By the Numbers, 60 Duke L.J. 789 (2010)⁠
    LIMIT

    A survey of written opinions, so it is evidence that these failures recur across courts and case types. It is not a population rate for productions generally. The Federal Judicial Center's report of this study restates the 401 as motions ruled on; the article's own unit is cases.

  6. 06
    FINDING

    90 of 230

    WHAT IT SAYS

    cases in which sanctions were awarded had failure to preserve ESI as the sole basis, and it was one of several types of misconduct in a further 46. Failure to produce was the sole basis in 35 cases and was mentioned in another 67 involving multiple types of misconduct.

    SOURCE Willoughby, Jones & Antine, 60 Duke L.J. 789, 803 (2010)⁠
    LIMIT

    Counts of the misconduct recorded in the opinions where sanctions were awarded. They are not a measure of how often either failure occurs. The article's abstract names one of the two as the most common basis and its body names the other, so the counts are given here and no ranking is.

  7. 07
    FINDING

    Higher recall and/or precision

    WHAT IT SAYS

    On NIST's legal-discovery test collection, a technology-assisted process yielded higher recall and/or precision than an exhaustive manual review. In the tested tasks the authors conclude recall at least as high as the official assessors' manual review, and higher precision.

    SOURCE Maura R. Grossman & Gordon V. Cormack, XVII Rich. J.L. & Tech. 11 (2011)⁠
    LIMIT

    Evidence against "a human looked at everything" as a completeness proof. Not a universal performance guarantee for every tool or matter, and not a claim about Absentia.

  8. 08
    FINDING

    Generally impossible

    WHAT IT SAYS

    NIST's TREC work notes that the ideal production, every responsive document and no non-responsive document, is generally impossible for a real ranking and cutoff.

    SOURCE NIST, Overview of the TREC 2010 Legal Track⁠
    LIMIT

    A statement about retrieval under a ranking and a cutoff. It does not say any particular production was deficient.

  9. 09
    FINDING

    "Best situated"

    WHAT IT SAYS

    The Sedona Conference states that responding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information.

    SOURCE The Sedona Principles, Third Edition, 19 Sedona Conf. J. 1 (2018)⁠
    LIMIT

    A consensus principle about who is best placed to make that judgment. It is not a rule of decision in any given court.

  10. 10
    FINDING

    A specific deficiency

    WHAT IT SAYS

    The challenger usually must point to a specific deficiency, or existing evidence supporting a reasonable deduction that other documents may exist, before obtaining discovery into the producing party's process. The authors write that the federal courts have not yet provided a clear standard for incomplete-production disputes.

    SOURCE Hon. Xavier Rodriguez & Hon. David L. Horan, Meta-Discovery: Allegations of an Incomplete Document Production, 19 Sedona Conf. J. 745 (2018)⁠
    LIMIT

    A survey of how courts have ruled. Practice varies by court and by matter.

  11. 11
    FINDING

    Reasonable inquiry

    WHAT IT SAYS

    Rule 26(g) requires a signature certifying, to the best of the signer's knowledge, information, and belief formed after a reasonable inquiry, that a disclosure is complete and correct as of the time it is made.

    SOURCE Fed. R. Civ. P. 26(g)⁠
    LIMIT

    A certification by the signer. It is not an independent audit of the production.

  12. 12
    FINDING

    After the loss

    WHAT IT SAYS

    Rule 37(e) applies to ESI that should have been preserved, was lost because reasonable steps were not taken, and cannot be restored or replaced. Remedies turn on prejudice, and the harshest measures require intent to deprive.

    SOURCE Fed. R. Civ. P. 37(e) and the 2015 Committee Note⁠
    LIMIT

    A remedy after loss. It does not detect the missing evidence.

Court disputes are the visible cases, not a failure rate for all productions.

02 THE LIMIT

There is no failure rate.

There is no defensible national failure rate for document productions. The system has no routine independent audit of what was produced. Courts publish only a fraction of disputes. Sanctions measure litigated misconduct rather than all misses.

Any statistic claiming that a fixed percentage of productions are incomplete would be invented. There is none on this site.

The case for a second test does not rest on a rate. It rests on who does the grading.

03 SOURCES

Sources.

  1. 1

    Federal Rule of Civil Procedure 26(g)

    GovInfo⁠
  2. 2

    Federal Rule of Civil Procedure 37(e) and 2015 Committee Note

    Cornell LII⁠
  3. 3

    Emery G. Lee III, Federal Judicial Center, Motions for Sanctions Based Upon Spoliation of Evidence in Civil Cases (2011)

    uscourts.gov⁠
  4. 4

    Dan H. Willoughby Jr., Rose Hunter Jones & Gregory R. Antine, Sanctions for E-Discovery Violations: By the Numbers, 60 Duke L.J. 789 (2010)

    Duke Law Scholarship Repository⁠
  5. 5

    Hon. Xavier Rodriguez & Hon. David L. Horan, Meta-Discovery: Allegations of an Incomplete Document Production, 19 Sedona Conf. J. 745 (2018)

    The Sedona Conference⁠
  6. 6

    The Sedona Principles, Third Edition, 19 Sedona Conf. J. 1 (2018)

    The Sedona Conference⁠
  7. 7

    Nicholas M. Pace & Laura Zakaras, RAND, Where the Money Goes (2012)

    RAND⁠
  8. 8

    Maura R. Grossman & Gordon V. Cormack, Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review, XVII Rich. J.L. & Tech. 11 (2011)

    Richmond JOLT⁠
  9. 9

    NIST, Overview of the TREC 2010 Legal Track

    NIST TREC⁠

Each figure on this site links to the source it was read from.