search

fgnt / sed_scores_eval

MIT License
26 stars 4 forks source link

Handling the case of a sample without any events. #1

Closed denfed closed 2 years ago

denfed commented 2 years ago

Hello, I have run into an issue with your codebase with a custom dataset I am running. I get a mismatch between the scores and ground truth dictionaries because some of my samples have no sound events at all, meaning that there isn't an entry for it in the ground truth csv. However that breaks the current codebase (io.py line 58).

My question is if this is breaking, how are you handling metric calculations for the cases of a sample not having any events? If there are no ground truth events and the model outputs some events of a class, that surely is a false positive.

Thanks.

denfed commented 2 years ago

I actually believe I figured it out. A sample that has no events should still be found in the ground truth csv, but its onset, offset, and event class should be np.nan.

JanekEbb commented 2 years ago

Hi Dennis,

thanks for your interest in our package.

yes, right. The example must still be listed in the ground_truth.tsv file with empty values for onset, offset, and event class. For an example you may have a look at the DESED validation set, e.g., the "YU0Cg_t_3TdI_30.000_40.000.wav" entry.

Hope that helps.

Best, Janek