search

LibEMG / libemg

MIT License
26 stars 9 forks source link

Divide by zero error in F1 Score calculation and calculation does not take into consideration rejected samples #87

Open lpetrich opened 1 week ago

lpetrich commented 1 week ago

In the _get_PREC_helper method it occasionally throws a divide by zero error on line 299precision[c] = tp / (tp + fp).

You can replicate this by running LDA on participant 16 from the 3DC dataset with a rejection threshold of 0.8 applied.

I believe the get_F1 should also ignore rejected samples, just like the get_CA method does, otherwise the recall score will drop from the -1 values giving a skewed F-score calculation.

ECEEvanCampbell commented 6 days ago

I agree that it makes sense that precision should ignore rejected samples, as rejected samples are neither true positive nor false positives. I'll add this issue to the kanban board in the projects tab to be done in the future.