May, 2024

Changjian Shui

<aside> 🔖 This ongoing blog will discuss my thoughts on characterizing responsible machine learning from a distribution shift perspective.

</aside>

Responsible machine learning (ML) refers to developing machine learning models by considering the broader societal impacts [ref1]. For instance, when developing a model, we should consider and avoid its potential negative societal impacts. These include potential malicious or unintended uses (e.g, disinformation, generating fake profiles), fairness considerations (e.g, the deployment of technologies that could unfairly affect specific subpopulations), privacy considerations and potential harms to human [ref2]. Specifically, responsible machine learning has been recently highlighted due to a vast progress in large language models (LLM) such as GPT [ref3].

Many recent research papers have proposed general ethical principles for monitoring and eventually building safeguards to prevent the abuse by bad actors (see Hinton recent talk **[ref4]). In this blog, I would like to highlight how to guarantee specific responsible ML aspects through the lens of distribution shift.

0. Distribution shift 101

First I’d like to give a brief introduction about distribution shift.

Concretely, we train a model $f$ from a distribution $P$ and aim to deploy it into a new distribution $Q$ with $P \neq Q$. In the general ML, we assume $P=Q$. Such a difference can lead to a performance degradation on $f$due to the distribution shift.

A simple demo of distribution shift. We train a model from the distribution P(x) and test into a new distribution P(x). [Image credit]

Below I’d like to talk about the implication of distribution shift in responsible machine learning.

1. Distribution shift in fairness

Algorithmic bias (or discrimination) is widely believed to be caused by the shifts among the subgroups of subpopulations [Ref5]. In plain language, if we consider subgroups by the gender and assume $P_\text{male} = P_\text{female} = P_\text{other genders}$, it’s easy to verify that no prediction discrimination occurs under various fairness criteria.

This immediately implies that any method in tracking distribution shift could potentially improve fairness. For example, my last blog summarized the relations between fairness and learning invariance in distribution shift.

Unique challenges in fairness While it may seem like the fairness issue has been addressed, there are still crucial elements that have been overlooked.

In the context of distribution shift, this cause is often easy to identify. E.g., the figure on the right side shows that the distribution shift is simply caused by the background change— from toy plane to real plane in the air. Therefore, it’s essential to learn an representation $Z$ that is independent of the source and target.

[Image Credit] In conventional distribution shift scenarios, the changing factor is rather simple such as background shift.

If we express it as an independence term, this implies $Z \perp A$ , where $A =$ (source, target). In contrast, in the fairness context, the variation factors $A$ could change significantly. Specifically, in intersectional fairness such as A = (sex, age, disability, education, many SDH factors), estimating and controlling the independence becomes non-trivial (see this paper and the following computational difficulties for details).