Alexanders101 / SPANet

Symmetry Preserving Attention Networks for Event Reconstruction
BSD 3-Clause "New" or "Revised" License
37 stars 36 forks source link

Add focal loss option for classification #9

Open jmduarte opened 1 year ago

jmduarte commented 1 year ago

Note I explicitly keep the current implementation F.cross_entropy if self.options.classification_focal_gamma == 0 out of an abundance of caution even though I checked that, numerically, the more complex formula gives the same answer.

The difference is that F.cross_entropy fuses F.log_softmax and F.nll_loss, so it is supposed to be more numerically stable, which I wanted to keep. But if you think it's unnecessary, we can remove this if-then clause.

@mstamenk

jmduarte commented 1 year ago

@Alexanders101 is this PR ok for you? Can you merge it? Or do you have any requested changes?