Add SNR calculation for signal fidelity

[matthewreyna]

I implemented a signal-to-noise (SNR) calculation for signal fidelity section of the benchmarks. The SNR is helpful for quantifying signal fidelity, including from quantization loss and different storage formats, e.g., signed 16-bit integers vs. other representations, and complements the (approximate) equality check from np.isclose.

The updated code successfully runs on my machine using the example command from the benchmark README:

./waveform_benchmark.py -r ./data/waveforms/mimic_iv/waves/p100/p10079700/85594648/85594648 -f waveform_benchmark.formats.wfdb.WFDBFormat516

With this example, the SNR is infinite, indicating that there is no "noise" in the signal.

Fidelity check:

Chunk           Numeric Samples               NaN Samples
    # Errors  /  Total    % Eq        SNR       NaN Values Match
Signal: II
  0              0/  51212896   100.000      inf            Y (7584)    
  1              0/     58944   100.000      inf            Y (1216)    
  2              0/   1818752   100.000      inf            Y (1408)    
Signal: III
  0              0/       960   100.000      inf            Y (320)     
Signal: V
  0              0/  51213248   100.000      inf            Y (7232)    
  1              0/     58944   100.000      inf            Y (1216)    
  2              0/   1818752   100.000      inf            Y (1408)    
Signal: aVR
  0              0/   3982176   100.000      inf            Y (1184)    
  1              0/  47229696   100.000      inf            Y (6144)    
  2              0/     58944   100.000      inf            Y (1216)    
  3              0/   1818752   100.000      inf            Y (1408)    
Signal: Pleth
  0              0/  25610240   100.000      inf             Y (0)      
  1              0/     29440   100.000      inf             Y (0)      
  2              0/    909440   100.000      inf             Y (0)      
Signal: Resp
  0              0/  12805120   100.000      inf             Y (0)      
  1              0/     15040   100.000      inf             Y (0)      
  2              0/    455040   100.000      inf             Y (0)    

With a small change to the example to simulate quantization, e.g., multiplying the signal by 0.99999 or 1.00001, the SNR is large but finite, indicating that there is a positive but small amount of noise that is not captured by the approximate equality check:

Fidelity check:

Chunk           Numeric Samples               NaN Samples
    # Errors  /  Total    % Eq        SNR       NaN Values Match
Signal: II
  0              0/  51212896   100.000     100.0           Y (7584)    
  1              0/     58944   100.000      99.9           Y (1216)    
  2              0/   1818752   100.000     100.0           Y (1408)    
Signal: III
  0              0/       960   100.000     100.0           Y (320)     
Signal: V
  0              0/  51213248   100.000     100.0           Y (7232)    
  1              0/     58944   100.000      99.9           Y (1216)    
  2              0/   1818752   100.000     100.0           Y (1408)    
Signal: aVR
  0              0/   3982176   100.000     100.0           Y (1184)    
  1              0/  47229696   100.000     100.0           Y (6144)    
  2              0/     58944   100.000      99.9           Y (1216)    
  3              0/   1818752   100.000     100.0           Y (1408)    
Signal: Pleth
  0              0/  25610240   100.000     100.0            Y (0)      
  1              0/     29440   100.000     100.0            Y (0)      
  2              0/    909440   100.000     100.0            Y (0)      
Signal: Resp
  0              0/  12805120   100.000     100.0            Y (0)      
  1              0/     15040   100.000     100.0            Y (0)      
  2              0/    455040   100.000     100.0            Y (0)  

Of course, as the noise increases, the SNR will decrease, but this should not occur unless there are large errors, e.g., comparing one signal with a different signal.

[briangow]

Thanks @matthewreyna , this looks good! I ran it across the waveform suite of files using WFDBFormat16.