This is reflective of the conflicting visions of fairness espoused by each mathematical definition and the underlying ethical assumptions. These toolkits can sometimes give practitioners conflicting information about an algorithm’s fairness, which is unsurprising given that it is mathematically impossible to meet some of the fairness conditions simultaneously. The fairness toolkit landscape so far reflects the reductionist understanding of fairness as mathematical conditions, as the implementations rely on narrowly defined fairness metrics to provide “pass/fail” reports. In line with this, a number of fairness toolkits Footnote 1 have been introduced, providing the means for testing the algorithm’s predictions against various fairness definitions. Dozens of metrics of fairness have been proposed, prompting efforts to disentangle their differences and rationale.

In response, there has been a proliferation of literature on algorithmic fairness aiming to quantify the deviation of their predictions from a formalised metric of equality between groups (e.g. The result is rising concern that these algorithmic predictions may be misaligned to the designer’s intent, an organisation’s legal obligations, and societal expectations, such as discriminating based on personal demographic attributes, e.g. These algorithms are using more data from non-traditional sources and employing advanced techniques in machine learning (ML) and deep learning (DL) that are often difficult to interpret. We propose Key Ethics Indicators (KEIs) as a way towards providing a more holistic understanding of whether or not an algorithm is aligned to the decision-maker’s ethical values.Īlgorithms are increasingly used to inform critical decisions across high-impact domains, from credit risk evaluation to hiring to criminal justice. In particular we highlight the debate around the acceptability of particular inequalities and the inextricable links between fairness, welfare and autonomy.

In this paper, we derive lessons from ethical philosophy and welfare economics as they relate to the contextual factors relevant for fairness. Moreover, fairness metrics tend to be implemented within narrow and targeted fairness toolkits for algorithm assessments that are difficult to integrate into an algorithm’s broader ethical assessment. However, these reductionist representations of fairness often bear little resemblance to real-life fairness considerations, which in practice are highly contextual.

Scholars have responded by introducing numerous mathematical definitions of fairness to test the algorithm, many of which are in conflict with one another. There is growing concern that decision-making informed by machine learning (ML) algorithms may unfairly discriminate based on personal demographic attributes, such as race and gender.

The Capital by A.H. Lee