Speaker
Description
We report on our progress in training PolarBERT, a foundation model for the IceCube Neutrino Observatory, and studying its generalization properties under domain shift induced by simulation imperfections.
The IceCube Neutrino Observatory at the South Pole consists of a cubic kilometer of Antarctic ice, instrumented with 5,160 digital optical modules. These modules collect light induced by neutrino interactions in the ice. This data is then used to identify the neutrino directions, energies, and types, which are essential inputs for both particle physics and astrophysics. Deep learning methods, such as graph neural networks, have been successfully applied to the steady stream of data that IceCube receives. In this work, we train a transformer-based foundation model on simulated IceCube data using a self-supervised learning objective, i.e. without relying on the true labels, that would otherwise need to be obtained from simulation. This is a first step towards being able to pre-train the model on real, unlabeled physics data. This pre-trained model can then be fine-tuned on various downstream tasks, such as directional reconstruction of neutrino events, in a sample-efficient manner.
In terms of performance, PolarBERT compares favorably to state-of-the-art supervised models while offering greater flexibility.
| Broad physics domain | Neutrino physics |
|---|---|
| AI/ML technique(s) to be presented | Foundation models, transformers, self-supervised learning |
