It seems self-evident that out of the hundreds of federal district court judges, some are better at their job than others. One possible measure of the quality of their work is the frequency with which their decisions are reversed on appeal. The defining feature of our court hierarchy is that higher courts have the power to correct lower courts. Thus, reversing the decision of a lower court means, in a sense, that the lower court judge made a mistake, whose correction delayed the adjudication of the case and incurred substantial appeals expenses.
Drawing and expanding on the methodology described by Royce DR Barondes in his 2010 article Federal District Judge Gender & Reflections, Westlaw Litigation Analytics appellate reports collected for 1052 district court judges commissioned between 1989 and 2020. After restricting to justices with at least 50 appeals in Westlaw statements and excluding very early decisions, some of which were most likely appeals of state court decisions, With 252,703 notes covering 877 judges. Due to Westlaw’s export restrictions, one judge with more than 1,000 notes has restricted the first 1,000 notes. After Barondes, “Inverse” is defined as Inverse, Partially Inverse, or Evacuated. In this series of posts, I will explore the dataset in a broad manner.
At first I must be clear that the reflection rate is an imperfect agent of quality. Sometimes a district court decision was already valid at the time, such as when a lower court judge correctly applied a precedent that is then overturned on appeal. Circuit courts also make mistakes, hence mechanisms such as re-hearing In the bench And appeal to the Supreme Court. However, more appeals are only appealed for a small fraction of appeals and many are finally confirmed. However, we must be aware that the Circuit Court’s decision is not the “basic fact” and that not all reversals are equal (consider a full reversal versus a partial reversal confirming every material aspect of the case with only a slight change in the calculation of damages).
For the scale, the graph of the number of views for each judge:
We can see that there are some outliers with a large number of appeals, but this is generally a fairly half-normal distribution, as would be expected given that a judge cannot have fewer appeals. This is the QQ plot versus a half-normal distribution:
With that in mind, let’s start at the most obvious place: the experiment. Are judges improving in their jobs over time? First, a graph of the number of notes according to the number of years that have elapsed since the judge was appointed.
(“Years since commission” is the number of years between a county judge’s assignment and the date of the appeal decision. I prefer to use the date of the lower court’s appeal appeal, but that was not available in Westlaw’s data.)
We see some positive skew here due to the fact that the judges leave the podium over time. There is also a clear “start-up” period of 0-2 years, primarily due to the lag between cases progressing enough to make a decision and then again to file an appeal.
We can now arrive at the main question: How does the rate of reflection vary by experiment?
The maximum data was excluded due to the lack of observations and the small number of judges. As Barronds found in 2010, there is a modest positive correlation between years on the bench and the reversal rate, our case. R2= 0.133. This is not enough by itself to show the importance, but this will await a more complete model.
One reason this is not sufficient to show significance is that experience does not exist in a vacuum. Experience only comes with advancing age, and age can also be associated with a change in reversal rates. First: the number of views according to age:
We can see that there is very little data outside of number 1Street And 99Y The percentages of ages 45 and 80. Looking at the age reversal rate by age shows a fairly clear relationship.
The reason for this can be many factors: Older judges may make bolder decisions, feel more confident in disagreeing with a fixed precedent, or it may reflect a loss of mental acuity. It is well known that there are differences in aging between men and women and this leads naturally to the question of whether there is a difference in rates of reversal as well, over time and in general. To learn more about it, the next article in this series will focus on sex.
Acknowledgments: I would like to thank Victoria Henige for asking the questions that led to this series of posts and Professor Lee Epstein To sharpen my thinking about it.
James Dealey is the Head of Legal Data Science at Skopos Labs, A legal research and analysis firm that provides a platform for machine learning and natural language processing of policy data. He is also a researcher at Washington University in St. Louis Center for Experimental Research in LawWhere he runs Supreme Court Database.