search

ExplainableML / czsl

PyTorch CZSL framework containing GQA, the open-world setting, and the CGE and CompCos methods.
GNU General Public License v3.0
111 stars 27 forks source link

ValueError: Harmonic mean only defined if all elements greater than zero #22

Closed jiayaohua closed 2 years ago

jiayaohua commented 2 years ago

in common.py 489line unseen_match the first element is 0 causes subsequent calculations of hm to be incorrect

mancinimassimiliano commented 2 years ago

Hello @jiayaohua! Yes, indeed if unseen_match is zero, the harmonic mean computed for the first element would be not determined. However, we used this implementation of the harmonic mean from scipy that tackles this issue by setting to 0 the HM if one of the two inputs is zero.

This is consistent with previous works (e.g. in CZSL and GZSL) and comes from the rationale that the HM in this context is measuring a tradeoff between two accuracies (i.e. seen/unseen): if one is zero, the tradeoff is not existent, thus we achieve the lowest possible value of the score (i.e. zero).

Finally, note that zero values do not affect the final computation of the HM: if for at least one bias value both unseen and seen accuracies are greater than zero, the output HM will be correctly computed and different from zero (as it is for all values we achieved, also when reproducing previous works).