chorus-ai / chorus_waveform

CHoRUS waveform documentation and various waveform conversion scripts
MIT License
3 stars 3 forks source link

Tidying, for the purpose of testing #42 #43

Closed tompollard closed 4 months ago

tompollard commented 4 months ago

This pull request is intended to test the github workflow that was introduced in #42. The changes introduced in this pull request are all minor formatting fixes (e.g. aligning comments, removing whitespace).

github-actions[bot] commented 4 months ago

Benchmark results:

Format: waveform_benchmark.formats.parquet.Parquet
         (Example format using Parquet)
Record: ./data/waveforms/mimic_iv/waves/p100/p10079700/85594648/85594648
         214981 seconds x 6 channels
         255177600 timepoints, 199126720 samples (78.0%)
________________________________________________________________
Channel summary information:
 signal       fs(Hz)     Bit resolution       Channel length(s)   
 II           249.89     0.005(mV)            212497              
 III          249.89     0.005(mV)            5                   
 V            249.89     0.005(mV)            212497              
 aVR          249.89     0.005(mV)            212492              
 Pleth        124.94     0.000244(NU)         212486              
 Resp         62.47      0.000244(Ohm)        212497              
________________________________________________________________
dynqmic row group size: 1249
dynqmic row group size: 1249
dynqmic row group size: 1249
dynqmic row group size: 1249
dynqmic row group size: 624
dynqmic row group size: 312
Output size:    394040 KiB (16.21 bits/sample)
Time to output: 12 sec
________________________________________________________________
Fidelity check:

Chunk        Numeric Samples          NaN Samples
    # Errors  /  Total    % Eq      NaN Values Match
Signal: II
  0              0/  51212896   100.000         Y (7584)    
  1      --- Different shapes (input: (60160,), file: (60159,)) ---
  2              0/   1818752   100.000         Y (1408)    
Signal: III
  0      --- Different shapes (input: (1280,), file: (1279,)) ---
Signal: V
  0              0/  51213248   100.000         Y (7232)    
  1      --- Different shapes (input: (60160,), file: (60159,)) ---
  2              0/   1818752   100.000         Y (1408)    
Signal: aVR
  0              0/   3982176   100.000         Y (1184)    
  1      --- Different shapes (input: (47235840,), file: (47235839,)) ---
  2      --- Different shapes (input: (60160,), file: (60159,)) ---
  3              0/   1818752   100.000         Y (1408)    
Signal: Pleth
  0              0/  25610240   100.000          Y (0)      
  1      --- Different shapes (input: (29440,), file: (29439,)) ---
  2      --- Different shapes (input: (909440,), file: (909439,)) ---
Signal: Resp
  0              0/  12805120   100.000          Y (0)      
  1      --- Different shapes (input: (15040,), file: (15039,)) ---
  2              0/    455040   100.000          Y (0)      
________________________________________________________________
Read performance (median of N trials):
 #seek  #read      KiB      sec     [N]
     0     -1        0  14.4946     [3] read 1 x 214981s, all channels
     0     -1        0   1.0507    [10] read 5 x 500s, all channels
     0     -1        0   9.0385     [3] read 50 x 50s, all channels
     0     -1        0  89.1610     [3] read 500 x 5s, all channels
     0     -1        0   2.8581     [5] read 1 x 214981s, one channel
     0     -1        0   0.1700    [57] read 5 x 500s, one channel
     0     -1        0   1.4304     [7] read 50 x 50s, one channel
     0     -1        0  14.3822     [3] read 500 x 5s, one channel
________________________________________________________________
tompollard commented 4 months ago

@briangow looks like the action is working. I switched back to the larger record (/data/waveforms/mimic_iv/waves/p100/p10079700/85594648/85594648) because this is the only one included in the repo. It takes about 7 minutes to run.

briangow commented 4 months ago

@tompollard , great! There can be a lot of variability in the read times from format to format. We can stick with the current record in the repo but I'm happy to add a shorter one if it becomes a problem.