search

art-daq / artdaq_daqinterface

Other
0 stars 1 forks source link

Update compressionLevel in simple_test_config #16

Closed eflumerf closed 2 years ago

eflumerf commented 2 years ago

This issue has been migrated from https://cdcvs.fnal.gov/redmine/issues/25674 (FNAL account required) Originally created by @eflumerf on 2021-03-24 20:09:08

Reports from SBN indicate that several different compression algorithms are now supported by ROOT, including ZSTD which has much better performance than other types while still achieving reasonable compression levels.

I propose replacing compressionLevel: 3 with compressionLevel: 501 by default in our simple_test_config configurations.

eflumerf commented 2 years ago

Comment by @eflumerf on 2021-04-05 14:06:22

Changes implemented on artdaq-utilities-daqinterface:feature/25674_UpdateCompressionLevel

eflumerf commented 2 years ago

Comment by @eflumerf on 2021-04-05 18:42:28

From SBN Slack: Gianluca Petrillo 8:34 PM The test: CPU time of RootOutput for the copy of a uncompressed 50-event data file from run 5292:

|. |\3. storage |_\3. CPU time | | Compression/level | file size | factor | vs. ZLib | time/event | factor | vs. ZLib | | no compression | 21.39 GiB | 1.0 | +175% | 0.79 s | 1.0 | -89% | | ZLIB/1 (101) | 7.79 GiB | 2.7 | - | 6.24 s | 8.8 | - | | LZ4/1 (401) | 12.11 GiB | 1.8 | +55% | 2.59 s | 3.7 | -58% | | ZSTD/1 (501) | 7.27 GiB | 2.9 | -7% | 2.86 s | 4.0 | -54% |

Depending on how much we can spare, we may still save half the storage. (edited)