Lenka Zdeborová among the winners of the ERC Advanced Grant

MARVEL member Lenka Zdeborová is among the recipients of the latest round of Advanced Grants from the European Research Center, which were just announced this week.

Zdeborová is associate professor at EPFL and head of the Statistical Physics of Computation Laboratory. Within MARVEL phase III, she is a group leader in Pillar 2,  Machine Learning Platform for Molecules and Materials.

She won the ERC grant for her project “Statistical physics of attention and sequence modelling with neural networks”.

The project aims to improve our understanding of how to train neural networks more efficiently—with less energy, using smaller datasets, and achieving greater robustness – by establishing solid theoretical principles. In particular, the project targets the attention mechanism, a key component of the “transfomer” architectures on which advanced AI models, such as those powering ChatGPT, are based. The theoretical understanding of learning with attention-based components and sequence models lags significantly behind. The project will address questions such as: How do AI abilities emerge with the scale of the neural network, viewing it as a type of phase transition? What is the minimal data and computational resources required to achieve a specific performance level? How can we theoretically justify the design choices in neural networks, considering data structure and computational efficiency constraints? The approach will leverage methods from the statistical physics of disordered systems, statistics, and probability to generate insights that can enhance the training algorithms and architecture design of future AI systems.

The Advanced Grant programme funds established, leading principal investigator who wants long-term funding to pursue a ground-breaking, ambitious project. 

A total of €721 million in Advanced Grants will be awarded to 281 researchers across Europe. This was the first time the call for proposals was open to researchers at Swiss host institutions since Switzerland was classified as a non-associated third country for Horizon Europe. Because these proposals were submitted in 2024, the research will be funded by Switzerland’s State Secretariat for Education, Research and Innovation.