Tutorial on
Advances in Bias-aware Recommendation on the Web

to be held as part of the 14th ACM International Conference on Web Search and Data Mining (WSDM2021)

March 8, 2021 8:30-13, GMT+2 - ONLINE

Introduction

Ranking and recommender systems are playing a key role in today's online platforms, definitely influencing the information-seeking behavior of tons of users. However, these systems are trained on data which often conveys imbalances and inequalities, and such patterns might be captured and emphasized in the results the system provides to the final users, creating exposure biases and providing unfair results. Given that biases are becoming a threat to information seeking, (i) studying the interdisciplinary concepts and problem space, (ii) formulating and designing a bias-aware algorithmic pipeline, and (iii) materializing and mitigating the effects of bias, while retaining the effectiveness of the underlying system, are rapidly becoming prominent and timely activities.

The proposed tutorial is organized around this topic, presenting the WSDM community with recent advances on the assessment and the mitigation of data and algorithmic bias in recommender systems. We will first introduce conceptual foundations, by surveying the state of the art and describing real-world examples of how a bias can impact recommendation algorithms from several perspectives (e.g., ethics and system's objectives). The tutorial will continue with a systematic presentation of algorithmic solutions to uncover, assess, and reduce bias along the recommendation design process. A practical part will then provide attendees with concrete implementations of pre-, in-, and post-processing bias mitigation algorithms, leveraging open-source tools and public datasets. In this part, tutorial participants will be engaged in the design of the bias countermeasures and in articulating impacts on stakeholders. We will finally conclude the tutorial with an analysis of the emerging open issues and future directions in this vibrant and rapidly evolving research area.

Target Audience

This tutorial is accessible to researchers, industry technologists and practitioners. For people not familiar with rankings, this tutorial covers necessary background material. No prior knowledge on bias is assumed. Basic knowledge of Python programming and of quite common libraries, such as Pandas and NumPy, is preferred but not strictly necessary.

After this tutorial, attendees will be able to understand key aspects of bias in personalized rankings, materialize biases into underlying systems, play with mitigation and articulate impacts on stakeholders, identify challenges and opportunities.

Outline

Due to the ongoing worldwide COVID-19 situation, the Bias tutorial will take place online on March 8, 2021, morning (8:30-13), UTC +02:00.

Timing Content
08:30 08:40 Welcome and Presenters' Introduction
08:40 10:00 Session I: Foundations
Recommendation Principles
  • Recommendation principles. To introduce the problems associated to algorithmic bias, we will present the recommendation task as the generation of the most effective personalized ranking for a user, as in modern recommender systems.
  • Multi-sided recommendation aspects. Recommender systems have an impact on multiple actors, namely consumers, providers, system's owners. We will present these actors and the phases of the recommendation process where they have a role (design, algorithm, and evaluation).
Hands on Recommender Systems
  • Data preparation starting from public datasets (i.e., COCO and Movielens datasets).
  • Model definition (e.g., user/item embeddings, layers stacking) and training (e.g., epochs, loss, optimizer)
  • User-item relevance matrix computation from a pre-trained model (e.g., model load, predictions).
  • Model evaluation oriented to utility (e.g., NDCG, beyond-accuracy metrics).
Algorithmic Bias Foundations
  • Motivating examples. We will present real-world examples where bias can impact recommendation, considering domains such as music, education, social platforms, and recruiting.
  • Perspectives impacted by bias. Bias has an impact on several perspectives such as the economy, law, society, security, technology, and psychology.
  • Ethical aspects influenced by bias. Bias can have an impact at the ethical level and lead to issues such as recommendation of inappropriate content, lack of privacy, violation of autonomy and identity, introduction of opacity, lack of fairness, or the compromising of users' social relationships.
  • Objectives influenced by bias. We will present recommendation objectives influenced by bias (utility, coverage, diversity, novelty, visibility, exposure) and provide examples of related work.
10:00 10:10 Coffee Break
10:10 11:40 Session II: Bias Mitigation
Bias through the Pipeline
  • Recommendation pipeline. We will provide an initial overview of the recommendation pipeline, to characterize how bias can exist at several stages, namely, data acquisition and storage, data preparation, model training, model prediction, model evaluation, and recommendation delivery.
  • Types of bias associated to the pipeline. We explore the types of bias that can emerge at different stages of the pipeline, i.e., those associated to the users, platforms, data collection, data preparation, model exploitation, and model evaluation.
Bias Mitigation Design
  • Bias-aware process pipeline. Intervention strategies to mitigate algorithmic bias require an analysis of where and how bias might affect the system. We present a pipeline to support mitigation design.
  • Techniques for bias treatment. We will present the three main classes of mitigation techniques (pre-, in-, and post-processing), along with examples of solutions proposed for recommender systems.
  • Real-world applications. We will present examples of real-world platforms and of their approaches to deal with bias.
Hands on Item Popularity Bias
  • Definition and characterization of item popularity biases in interactions and recommendations.
  • Application of mitigation techniques based on pre-, in-, and post-processing.
  • Comparison of mitigation techniques based on bias and recommendation utility trade-offs.
  • Comparison of mitigation techniques on beyond-utility metrics (e.g., coverage, diversity, novelty).
11:40 11:50 Coffee Break
11:50 13:20 Session III: Unfairness Mitigation
Discrimination through the Pipeline
  • Types of discrimination. When bias affects users' sensitive attributes, it may lead to discrimination. We present concepts, such as direct/indirect discrimination, its granularity (group, individual, and subgroup discrimination), types of disparity (disparate treatment, impact, and mistreatment).
  • Definitions of fairness. We will present definitions of fairness and different classes in which thy can be categorized (equalized odds, equalized opportunity, demographic parity, fairness through (un)awareness, equality of treatment).
  • Recommendation pipeline. We will extend the discussion on the recommendation pipeline in terms of how bias can lead to unfairness during data acquisition and storage, data preparation, model training, model prediction, model evaluation, and recommendation delivery.
Unfairness Mitigation Design
  • Bias-aware process pipeline. Intervention strategies to mitigate algorithmic unfairness require an analysis of where and how bias might affect unfairness in a system. We present a pipeline to support mitigation design.
  • Techniques for unfairness treatment. We will present the three main classes of mitigation techniques (pre-, in-, and post-processing), along with examples of solutions proposed for recommender systems.
  • Real-world applications. We will present examples of real-world platforms and of their approaches to deal with unfairness.
Hands on Item Provider Fairness
  • Association of items to providers and sensitive attributes.
  • Characterization of providers representation in the catalog and in the interactions.
  • Identification of minority providers, both at individual and group level.
  • Definition and measurement of item provider unfairness on recommendations.
  • Application of mitigation techniques based on pre-, in-, and post-processing.
  • Comparison of mitigation techniques based on fairness and recommendation utility trade-offs.
  • Comparison of mitigation techniques on beyond-utility metrics (e.g., coverage, diversity, novelty).
13:20 13:30 Challenges, Final Remarks, and Discussion

Material

Presenters

Ludovico Boratto

Ludovico Boratto
EURECAT - Centre Tecnòlogic de Catalunya (Spain)

Ludovico Boratto is senior research scientist in at EURECAT. His research focuses on recommender systems and on their impact on stakeholders. His research has been published in top-tier conferences and journals. He is editor of the book "Group Recommender Systems: An Introduction" (Springer). He is editorial board member of the "Information Processing & Management" journal (Elsevier) and guest editor of other special issues. He is regularly PC member of the main Data Mining conferences. In 2012, he got a Ph.D. at the University of Cagliari, where he was research assistant until May 2016.



Mirko Marras

Mirko Marras
École Polytechnique Fédérale de Lausanne EPFL (Switzerland)

Mirko Marras is a Postdoctoral Researcher at the École Polytechnique Fédérale de Lausanne EPFL. His research focuses on data mining and machine learning for recommender systems, with attention to bias issues, mainly under online education settings. He authored papers in top-tier journals, such as Pattern Recognition Letters and Computers Human Behavior. He gave talks and demos at international conferences and workshops, e.g., TheWebConf2018, ECIR2019, and INTERSPEECH2019. He is PC member of major conferences, e.g., ACL, AIED, EDM, ECML-PKDD, EMNLP, ITICSE, ICALT, UMAP. He co-chaired the BIAS2020 workshop at ECIR2020 and gave a tutorial on Bias in Recommender Systems at UMAP2020 and ICDM2020. In 2020, he received a Doctoral Degree from University of Cagliari.

Registration

The registration to the tutorial is managed by the WSDM Main Conference through the Registration Portal.

You need to opt for a one-day registration fee and then select this workshop along the registration process. If you need any other information, please do not hesitate to contact us.

Contacts

Please, reaching out to us at ludovico.boratto@acm.org and mirko.marras@epfl.ch.

Past Editions

We also invite you to check out previous editions of our similar tutorials: