Homepage of Aditya Krishna Menon

Staff Research Scientist

Google

New York, NY

Email: adityakmenon followed by google.com

Biography

I'm a Staff Research Scientist at Google. I work on machine learning and its applications.

I completed my honours in Computer Science from the University of Sydney in 2006 under Sanjay Chawla. I completed my PhD in Computer Science from UC San Diego in 2013 under Charles Elkan. From 2013 - 2018, I held positions at NICTA, CSIRO Data61, and the Australian National University, working with Bob Williamson and other inimitable colleagues.

Here is a copy of my CV.

Research interests

I am broadly interested in the design and analysis of machine learning algorithms for (weakly) supervised learning problems occurring in practice. Specific areas of interest include:

Weakly-supervised learning (e.g. learning from label noise, positive and unlabelled learning)
Classification with real-world constraints (e.g. class imbalance, fairness)
Relations amongst foundational problems (e.g. class-probability estimation, bipartite ranking)

Selected publications

Below are a few representative publications. For a full list, see here.

Inspired by Bernard Chazelle's wonderful idea of "liner notes" for his papers, I've included some of my own for a few papers. (Liner notes are not peer-, or even coauthor-reviewed.)

Long-tail learning via logit adjustment.
Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar.
In International Conference on Learning Representations (ICLR), 2021.
[pdf]
Can gradient clipping mitigate label noise?
Aditya Krishna Menon, Ankit Singh Rawat, Sashank J. Reddi, and Sanjiv Kumar.
In International Conference on Learning Representations (ICLR), Addis Ababa 2020.
[pdf]
Fairness risk measures.
Robert C. Williamson and Aditya Krishna Menon.
In International Conference on Machine Learning (ICML), Long Beach, 2019.
[pdf]
Bipartite ranking: a risk-theoretic perspective.
Aditya Krishna Menon and Robert C. Williamson.
In Journal of Machine Learning Research (JMLR), Volume 17, Issue 195. 2016.
[pdf] [liner notes]

Back in 2011, months of exhaustive experimentation with AUC maximisation in the context of link prediction taught me one thing: it's hard to do better than logistic regression. I was excited when later that year, Kotłowski et al. gave a compelling theoretical explanation for why this might be.

When I joined NICTA in 2013, I followed a rite of passage and read Mark Reid and Bob Williamson's tour de force on information, divergences, and risks. Tucked away in that paper was a mention of how the AUC related to the concepts presented. I innocently mentioned to Bob that perhaps there was more to be said here. He agreed, and suggested that I start by spelling out what was in my mind.

We never expected it would take quite that long to spell. [hide]
Linking losses for density ratio and class-probability estimation.
Aditya Krishna Menon and Cheng Soon Ong.
In International Conference on Machine Learning (ICML), New York City, 2016.
[pdf] [slides] [poster] [code] [liner notes]

Long before being introduced to proper losses, I'd known about, and completely accepted, the basic premise as to why density ratio was different to class-probability estimation: the later only indirectly modelled the ratio, which is generally a sub-optimal strategy. Sometime in late 2015, I re-read the cool work on LSIF. This time, with link functions and partial losses fresh in my mind, I noticed that one could think of this as involving a particular proper loss plus link function. This was mildly interesting, but the point remained: using anything but the link that directly yields the density ratio must be a bad idea.

This was something of a defeat for my running theory at the time, that most everything can be attacked with some version of logistic regression. To better understand this failure mode, I was looking at Sugiyama's elegant unified Bregman view of density ratios. I noticed that logistic regression was mentioned, but didn't pay much attention to it; after all, the Bregman view of regret under proper losses immediately led to the KL minimisation view of logistic regression.

It was only on a re-read that I realised there was a bit more to this innocuous result: it was a statement of Bregman minimisation not to the true probability, but to the true density ratio. Now that I didn't know was true for logistic regression. Studying the surprisingly simple proof, I saw it could be viewed as a surprising equality between the KL divergence on probabilities and on ratios.

I was sure this couldn't be true in general; but why? I worked through the case of a general divergence, trying to find out the line where the proof would break.

Half an hour later, after staring at my working multiple times, I was still incredulous that everything seemed to work in general, too. And so was Lemma 2 born. [hide]