BY MATT HUTCHINS
 |
 |
Yale Professor Ian Ayres spoke in Austin Hall on Wednesday, warning researchers that statistical studies of disparate impact should avoid the inclusion of variables without a plausible justification. The central hypothesis of his presentation was that in a study of disparate impact the inclusion of control variables which are causal but inappropriate may lead to understatement of racial bias. A disparate impact study differs from a normal disparate treatment study because rather than attempting to narrowly expose the causal strength of race in an allegedly discriminatory process, the analysis aims at instead at recognizing the impact of applying the same decision-making process to individuals of different race.
Ayres provided numerous examples to support his hypothesis, beginning with the consideration the use of high school diplomas in Griggs v. Duke Power Co. by the employer as a proxy for race. Ayres argued that by statistical measures, a study of the impact of Duke Power’s policies that controlled for the influence of a high school diploma would not find any independent disparate racial impact.
Nonetheless, the high school variable, when excluded, clearly has a close connection with race and discriminatory impact. Just like the high school diploma variable, controlling for some variables that may otherwise be causal could result in an understatement of disparate impact.
Ayres applied this theory to a recent Los Angeles Police Department study of traffic stops in a paper that was coauthored by HLS 1L Jonathan Borowski. He found that there were some control variables which had been included in the LAPD’s analysis that were not plausibly justified and that caused a significant understatement of the disparate impact on minorities.
The LAPD gathered data on every traffic stop conducted by the force in a year and provided it to a team of professional analysts, who found that when there was a traffic stop, blacks were 51.18 % more likely than whites to be asked if they could be searched and 21.39 % more likely to be arrested; Hispanics were 3.96 % more likely than whites to be asked for a search and 28.53 % more likely to be arrested. One key set of variables Ayres identified was the characteristics of the officer, including age, sex, and number of complaints against an officer and commendations. Although these variables might have some causal relation to the behavior of the officer and individual stopped, Ayers argued that the inclusion of these variables as controls cannot be plausibly justified. When excluded from the regression analysis, the numbers for the “Black Effect” increased to 29.15% and 76.36% respectively, and the “Hispanic Effect” rose to 32.35% and 15.95%. One critical difference is that the increase of the “Hispanic Effect” given the exclusion of these variables is not only of a magnitude of over 400 %, it also resulted in a T-statistic of 7.62 rather than 1.54, meaning that the correlation could be seen as statistically significant. This sort of difference could have a major impact on the success or failure of disparate impact civil rights litigation.
Another technique advocated by Ayres was “capped coefficient” analysis, where a variable which is plausibly included is limited in its potential effect on the regression because of the disproportionate bias it introduces. In Ayres’ example of auto loan origination, the variable of “no doc” loans that are originated without the full documentation of the applicant, may plausibly be considered to introduce higher costs for the originator. At the same time, this variable may be highly correlated with race and socioeconomic factors. A reasonable estimate of the additional cost of originating a no doc loan can be made by contacting financial experts, and even when such allowances are made generously in favor of the loan originators, it is likely that the inclusion with proper caps will reduce the understatement of disparate racial impact relative to the non-capped analysis. This is particularly evident in light of the estimate that retail mark-ups of auto loans for minority applicants are twice as high as for white applicants and “no doc” correlations related to more than three times a reasonable estimate of additional costs.
The third area in which Ayres advocated a different approach to analysis is Outcome Testing. Instead of controlling for numerous variables, an outcome test analysis aims at understanding how, given a certain decision, a model would predict other variables. Ayres provided the example of a study by Kumar & Wolfers, “Underestimating Female CEOs”, which showed the phenomenon as it appears in the systematic “surprise success” of female CEOs. It turns out that male analysts, on average, provide predictions of a female CEO’s performance which are disappointed 7% less often than the predictions of male CEOs and outperformed 7.7 % more often than by males. The predictions by female analysts of the same female CEOs do not result in the same phenomenon. Thus, simply by predicting the outcome, with no control variables, there are cases in which the disparate impact which is incorporated into a decision maker’s actions can be revealed.The scholars in the audience were receptive to Ayres’ proposals and yet skeptical of the ability to apply them in a straightforward way, given the many ways in which he has rejected the universal applicability of traditional rules of thumb. The techniques Ayres has outlined may be useful in the future for uncovering areas where prior studies had understated disparate impact, but it will take time before his ideas gain support throughout the community of empirical analysts.