search

chorus-ai / chorus_waveform

CHoRUS waveform documentation and various waveform conversion scripts
MIT License
3 stars 3 forks source link

Waveform Benchmark Crashes When Format's Nan Locations Aren't Identical To Expected #108

Open WilliamDixon opened 1 day ago

WilliamDixon commented 1 day ago

Describe the bug line 378 and line 440 in benchmark.py remove nan values from the format's output from read_waveforms and open_read_close_waveforms based on a nan mask created from the expected data. A few lines later 386 and 448 compute_snr(data_nonan, filedata_nonan) is called which is designed to crash if filedata_nonan has any nan values (which it will if the format being tested adds nans that the benchmark doesn't expect).

To Reproduce Alter one of the formats so the first (or any) value of its output array is set to np.nan instead of whatever its supposed to be, and run the benchmark code.

Expected behavior The benchmark process shouldn't crash, it should output in the fidelity report that there was 1 more nan value than expected.

matthewreyna commented 1 day ago

We can safely remove the assert for finite values in the signals from the compute_snr function in benchmark.py: https://github.com/chorus-ai/chorus_waveform/blob/main/waveform_benchmark/benchmark.py#L157

The SNR will be undefined, i.e., the function will return a NaN and a warning, but it will not crash.