Closed tompollard closed 4 months ago
Benchmark results:
Format: waveform_benchmark.formats.parquet.Parquet
(Example format using Parquet)
Record: ./data/waveforms/mimic_iv/waves/p100/p10079700/85594648/85594648
214981 seconds x 6 channels
255177600 timepoints, 199126720 samples (78.0%)
________________________________________________________________
Channel summary information:
signal fs(Hz) Bit resolution Channel length(s)
II 249.89 0.005(mV) 212497
III 249.89 0.005(mV) 5
V 249.89 0.005(mV) 212497
aVR 249.89 0.005(mV) 212492
Pleth 124.94 0.000244(NU) 212486
Resp 62.47 0.000244(Ohm) 212497
________________________________________________________________
dynqmic row group size: 1249
dynqmic row group size: 1249
dynqmic row group size: 1249
dynqmic row group size: 1249
dynqmic row group size: 624
dynqmic row group size: 312
Output size: 394040 KiB (16.21 bits/sample)
Time to output: 12 sec
________________________________________________________________
Fidelity check:
Chunk Numeric Samples NaN Samples
# Errors / Total % Eq NaN Values Match
Signal: II
0 0/ 51212896 100.000 Y (7584)
1 --- Different shapes (input: (60160,), file: (60159,)) ---
2 0/ 1818752 100.000 Y (1408)
Signal: III
0 --- Different shapes (input: (1280,), file: (1279,)) ---
Signal: V
0 0/ 51213248 100.000 Y (7232)
1 --- Different shapes (input: (60160,), file: (60159,)) ---
2 0/ 1818752 100.000 Y (1408)
Signal: aVR
0 0/ 3982176 100.000 Y (1184)
1 --- Different shapes (input: (47235840,), file: (47235839,)) ---
2 --- Different shapes (input: (60160,), file: (60159,)) ---
3 0/ 1818752 100.000 Y (1408)
Signal: Pleth
0 0/ 25610240 100.000 Y (0)
1 --- Different shapes (input: (29440,), file: (29439,)) ---
2 --- Different shapes (input: (909440,), file: (909439,)) ---
Signal: Resp
0 0/ 12805120 100.000 Y (0)
1 --- Different shapes (input: (15040,), file: (15039,)) ---
2 0/ 455040 100.000 Y (0)
________________________________________________________________
Read performance (median of N trials):
#seek #read KiB sec [N]
0 -1 0 14.4946 [3] read 1 x 214981s, all channels
0 -1 0 1.0507 [10] read 5 x 500s, all channels
0 -1 0 9.0385 [3] read 50 x 50s, all channels
0 -1 0 89.1610 [3] read 500 x 5s, all channels
0 -1 0 2.8581 [5] read 1 x 214981s, one channel
0 -1 0 0.1700 [57] read 5 x 500s, one channel
0 -1 0 1.4304 [7] read 50 x 50s, one channel
0 -1 0 14.3822 [3] read 500 x 5s, one channel
________________________________________________________________
@briangow looks like the action is working. I switched back to the larger record (/data/waveforms/mimic_iv/waves/p100/p10079700/85594648/85594648) because this is the only one included in the repo. It takes about 7 minutes to run.
@tompollard , great! There can be a lot of variability in the read times from format to format. We can stick with the current record in the repo but I'm happy to add a shorter one if it becomes a problem.
This pull request is intended to test the github workflow that was introduced in #42. The changes introduced in this pull request are all minor formatting fixes (e.g. aligning comments, removing whitespace).