Understanding and generalization beyond the training distribution are regarded as huge challenges in modern machine learning (ML) — and Yoshua Bengio argues it’s time to look at causal learning for possible solutions. In the paper Towards Causal Representation Learning, Turing Award honoree Bengio and his research team make an effort to unite causality and ML research approaches, delineate some implications of causality for ML, and propose critical areas for future research.
Bengio outlined the challenge in a causal representation learning talk he gave late last year, “I would say there are pretty significant gaps between current state-of-the-art in machine learning-driven AI and the intelligence that we see deployed in humans and many animals… We don’t have AI systems that actually understand at the level that humans do, or anywhere close.” Bengio characterized the meaning of human-level AI “understanding” as: capture causality; capture how the world works; understand abstract actions and how to use them to control; reason and plan, even in novel scenarios; explain what happened (inference, credit assignment); and generate out-of-distribution.
In this regard, most modern ML models remain far from true understanding, as they work only under fixed experimental conditions and interventions in the real world are seen as a nuisance that can hopefully be engineered away. It is therefore not surprising that most of today’s ML models lack an out-of-distribution generalization ability.
Causal learning, on the other hand, focuses on representing structural knowledge about the data-generating process to allow interventions and changes, making it easier to re-use and re-purpose learned knowledge. This approach is considered closer to human thinking.
This article was originally published at Synced.
Regarding to AI and Causal Inference, Professor Judea Pearl is a distinguished pioneer for developing a theory of causal and counterfactual inference based on structural models. In 2011, Professor Pearl won the Turing Award. In 2020, Michael Dukakis Institute for Leadership and Innovation (MDI) and Boston Global Forum (BGF) also awarded Professor Pearl as World Leader in AI World Society (AIWS). At this moment, Professor Judea is a Mentor of AIWS.net and Head of Modern Causal Inference section, which is one of important AIWS.net.
