For my final-year project I built EngagePredict, a model that predicts how engaging a talk will be from its transcript and audio. It looked at semantic, sentiment and prosodic features across more than 2,300 samples and 450 hours of TED talks, and I compared 10 different models.
The first version scored suspiciously well. Not a little better than I expected, but well enough that it didn't make sense for such a subjective target.
What was actually going on
When I dug in, the label turned out to be reconstructable from the features themselves. The model wasn't learning what makes a talk engaging; it was partly just reading the answer back. That's label leakage, and it's easy to miss when the results look good.
I withdrew that version and rebuilt it honestly. It was not a fun week, but a model that only works because the answer is hidden in its inputs isn't a model.
Turning it into a tool
The experience made me wonder how often this happens in datasets people don't build themselves. So I built LeakGuard, a screen you can run on any dataset to check whether its label can be rebuilt from its own features, point to which features are responsible, and remove them.
The core idea is simple. If a label is really just a formula of the columns, a plain linear model recovers it almost perfectly. So LeakGuard compares how well simple models do against more flexible ones, and a suspicious gap is a red flag. That score is calibrated against a few deliberately leaky "anchor" labels built on the same data. And if the data can't support a clear answer, LeakGuard says so instead of guessing.
Where it is now
The work turned into a paper, "Detecting and Localising Label Leakage in Benchmark Datasets", where I'm the first author. It's under review right now. The biggest lesson for me: when a result looks too good, that's the moment to be most curious, not the moment to stop.