ParticleTransformer training for decay mode classification is unstable

HEP-KBFI / ml-tau-en-reg

Reconstructing tau leptons using various machine learning techniques

GNU General Public License v3.0

0 stars 1 forks source link

Closed jpata closed 4 months ago

jpata commented 4 months ago

As seen here: https://github.com/HEP-KBFI/ml-tau-en-reg/pull/21#issuecomment-2129914582: The validation loss of ParticleTransformer for training_type=dm_multiclass fluctuates like crazy. Possible options:

check what is different between predictions from epoch to epoch that introduces large fluctuations
merge some rare decay modes so the classes are better represented
increase dropout
lower lr even further

jpata commented 4 months ago

Fixed in #23.