New Data Undermine Claims of Institutional Bias at the PTAB

Last year, RPX published a critique of an August 2020 amicus brief filed by US Inventor, Inc. (“USI”) in New Vision Gaming & Development v  S.G. Gaming. The USI brief claimed to have found evidence of bias in the institution of America Invents Act (AIA) trials at the Patent Trial and Appeal Board (PTAB) from 2013 through 2018.

Specifically, USI claimed to have identified a statistically significant “October Effect” whereby the PTAB’s Administrative Patent Judges (APJs) are allegedly more likely to institute trial on less meritorious (or “questionable”) petitions in October than in September. USI claimed that October is significant because it is the first month of the USPTO’s fiscal year, and it suggested that APJs are “stuff[ing] the pipeline” with “questionable” petitions at the beginning of each fiscal year to “guarantee . . . future work” and to “please their budget-minded bosses”.

Underpinning USI’s brief (the “USI Study”) was its so-called “Questionable Institution Ratio” (“QIR”), a statistic that purports to measure the relative number of trial institution decisions that are “questionable” based on the final outcome of those trials. RPX examined USI’s results in detail, making certain adjustments designed to compensate for methodological errors and other defects, and also tested USI’s hypothesis using a fuller dataset. [1] That analysis found that the USI study failed to support a conclusion of bias at the PTAB with any statistical significance.

The USI Study considered institution decisions made by the PTAB in September and October of each year from 2013 through 2018, with 2018 being the last year in which reasonably complete trial outcome data were available. (The PTAB’s Final Written Decisions, or “FWDs”, generally come approximately twelve months after the institution decision.)

Now—seven months later—complete data are available for PTAB trials instituted in September and October of 2019.

Figure 1 shows the QIR with the adjustments described in the earlier RPX analysis. [2] Based on this latest data, the October QIR exceeds the September QIR in only four out of seven years. [3]

The updated data clearly show that no October Effect exists. Not only did USI’s October Effect fail to appear in 2019, a handful of FWDs from 2017 saw their outcomes change on remand  from the Federal Circuit, causing the October Effect from 2017 to disappear as well—underscoring the fragility of USI’s conclusion about statistical significance based on so few datapoints.

Moreover, with the addition of the 2019 data, the aggregate number of “questionable” institution decisions in the seven years through 2019 was nearly the same for institution decisions made in September and October (52 vs. 53). As noted above, USI purported to provide evidence supporting the argument that the PTAB and its APJs derive an economic benefit from instituting questionable trials because the PTAB keeps the trial portion of a petitioner’s filing fee when a trial is instituted but refunds the fee when a trial is not instituted. The differential benefit in seven years of Octobers vs. the corresponding September months comes to a mere $22.5K. To put that in context, AIA trial institution fees come to $23M per year, according to New Vision Gaming’s brief at the Federal Circuit.

As RPX observed in its earlier study, USI mistook random variation—or “noise”—for a signal that doesn’t exist. Figure 2 shows the average QIR for institution decisions made by month from 2013 to 2019. The adjusted QIRs for each of the twelve calendar months range from 13.6% to 24.1% with an average of 18.1%. The QIR for September (17.4%) and October (18.1%) fall in the middle of that range, September being slightly lower than the overall average and October being exactly equal to the overall average.

USI’s claim of an October Effect illustrates the dangers of drawing conclusions on small sets of data while ignoring the larger context. USI based its claims of a statistically significant October Effect on only six pairs of datapoints, while ignoring the possibility that the observed data were within the normal month-to-month variation in the statistic they measured. This deficiency was compounded by a number of questionable definitional and methodological decisions that are covered in RPX’s earlier critique.

Ultimately, the most remarkable thing about the institution decisions made in September and October is just how unremarkable they are.

[1] In the USI Study, USI’s definition of “questionable” did not align with the statutory threshold for institution set by the AIA. Specifically, USI classified institution decisions that resulted in a mixed FWD as “questionable” even though the standard is “a reasonable likelihood that the petitioner would prevail with respect to at least 1 of the claims challenged in the petition” 35 U.S.C. § 314(a). Further, USI’s choice of denominator inflated the QIRs and exaggerated the differences between months.

To address these issues, the QIR was adjusted from USI’s original calculation, (claims patentable or mixed outcome) / (all claims patentable or all claims amended), to (as all claims patentable over) / (all claims patentable + all claims amended, or mixed outcome). See the original RPX critique for details.

[2] Figure 1 replicates figure 7 of the original article with the latest available data.

[3] We note that institution decisions made in September and October of 2019 pre-dated publication of the USI study by nearly one year, so the PTAB’s institution decisions in these cases could not have been influenced by the USI study.