Is the university application system biased?

A number of researchers have argued that there is evidence of bias against Black, minority and ethnic (BME) university applicants. The Universities and Colleges Admissions Service (UCAS) has repeatedly refuted these claims. But UCAS has, until very recently, refused to publish the full application data for researchers to examine. This blog examines the existing and latest evidence of whether there is bias against BME students applying for university.

Background

Last September, UCAS published a short data release which they argued showed there was no systematic bias against BME applicants in the university application system. While the offer rate for BME applicants was 15 percentage points lower than for white applicants, UCAS argued that this was attributable, almost entirely, to differences in applicants’ predicted A-level grades. That is, a BME applicant predicted to achieve three A grades in their A-Level examinations is as likely to be offered a place at university as a white applicant predicted to achieve the same grades.

However, a number of researchers criticised the UCAS data release. Vikki Boliver, a senior lecturer at Durham University, suggested that the UCAS data was incomplete. Boliver observed that the UCAS data, released in September, excluded applicants to the Universities of Cambridge and Oxford, excluded applicants to medicine and dentistry courses, and excluded applicants who were predicted to achieve three A* grades at A-Level. In addition, she criticised UCAS for amalgamating ethnic minorities into one, single group. She argued that some ethnic minority groups were more likely to experience bias than others, and that amalgamating the groups might conceal this.

Boliver pointed to her own research that shows that predicted grades only partially account for the lower offer rates for BME applicants. Indeed, her evidence suggests that the bias against BME applicants is greater when the applicant is applying to a course which has a large proportion of BME applicants. She argues that this suggests that admissions selectors may be rejecting some of their BME applicants to achieve a more ethnically representative student body.

Other evidence has supported Boliver’s findings. Academics from the London School of Economics and the University of Bristol analysed UCAS data from 2008. They controlled for a number of variables which sought to capture the academic quality of applicants, such as A-Level results and UCAS tariff score. They found that, when controlling for academic quality, BME applicants from almost every different group received significantly lower offer rates in all subject areas than white applicants; only mixed-race applicants obtained a similar offer rate to white applicants. This study also carried out a further analysis which controlled for social characteristics (social class background, gender and school type). When controlling for social characteristics, the researchers found that the lower offer rates persisted for all BME applicants except, again, mixed race applicants.

New data

Three weeks ago, after significant Government pressure, UCAS published a large swathe of application data broken down by ethnicity. Since then a number of analyses of the data have been published. It should be noted that these early analyses will almost certainly be surpassed by more robust academic evaluations in the coming months and years. Furthermore, as Mark Leach, a former advisor to the Shadow Minister for Universities & Science and editor of WonkHE, has stated, large releases of data can lead to information overload and create a risk of misinterpretation. We should be wary of this when considering these analyses.

However, the early analyses can still provide an interesting insight into the bias debate.  The data released by UCAS provides application and offer data at the institutional level. The data provides an ‘offer rate’ (the proportion of applicants who are offered a university place) which is broken down by sex, ethnicity and socio-economic background. The rate controls for predicted grades.

Across the whole sector, the data appears to show there is no statistically significant difference in offer rates between white applicants and all BME applicants. There are two significant caveats to this.

First, while UCAS argues that their method is the “most precise” they have, so far, refused to publish the underlying data which has allowed them to develop this method. Second, the Equality Challenge Unit have argued that small, non-statistically significant differences in offer rates can be projected into much greater differences in the graduate labour market and in postgraduate opportunities. To illustrate this, they offer an example from a different context, the promotion of women within US companies. Research shows that women receive less favourable evaluation of their work than men. This research suggests that the bias is between 1% and 5%. While this appears a small difference, computer simulation illustrates the larger effects this can have on the promotion of women over time. In a cohort of 500 people, a 5% bias in evaluation would lead to 29% of the highest-level staff (the most promoted) being women and 58% of the lowest level staff (the least promoted) being women. Similarly, a small bias against BME candidates may lead to considerable differences later on.

While the data shows no difference in offer rates across the sector, there appears to be evidence of some bias at an institutional level. WonkHE’s analysis has shown that 28 institutions had a significant gap in the ‘offer rate’ between Black (which is comprised of Black – Caribbean, Black – African, Black – other, Black or Black British – Caribbean, Black or Black British – African and applicants of other Black background) applicants and the average application rate, after controlling for predicted grade, in, at least, two of the last three years. Similarly, 25 institutions had a significant offer rate gap for Asian applicants (which is comprised of Asian – Indian, Asian or Asian British – Indian, Asian – Pakistani, Asian or Asian British – Pakistani, Asian – Bangladeshi, Asian or Asian British – Bangladeshi, Asian – Chinese, and applicants of other Asian background). In both cases, this represents around one fifth of higher education providers in the UK. Interestingly, particularly in regards to the offer rate gap for Asian applicants, many of the offending institutions are those with the most Asian applicants. This supports Boliver’s theory that admissions selectors may reject some of their BME applicants to achieve a more representative student body.

Conclusion

There has been a longstanding debate over whether there is bias in the university admissions system. UCAS has recently published a large quantity of data regarding offer rates and the ethnicity of applicants. Early interpretations of the data suggest that across the whole sector there is little evidence of bias against BME applicants. However, we should be cautious about over-interpreting these early analyses. Academics are likely to conduct more robust examinations of this data that may show different findings. At an institutional level, there is some evidence of bias particularly among universities which have a large proportion of BME applicants. The Government should be congratulated on persuading UCAS to publish this data which has provided much more transparency to the bias debate.

James Dobson is a researcher at Bright Blue