Open peter-madigan opened 6 years ago
Repeated with 100,000 hits and it fails to sort correctly on 133 packets.
Further inspection of the file shows that there are some chunks of high rate data in which large numbers of packets are generated with very small differences in time (likely a bad channel that then fills the chips' fifo). This can be simply overcome by increasing the sorted buffer to ~2000 hits at the cost of performance.
This has been tested successfully on first 100,000 hits of datalog_2018_04_15_13_27_44_CEST_.h5
.
In the future, we may want to consider fixing this more permanently (e.g. preprocess files with a sorting algorithm).
Commit 11a188d97b80a7ffbf67f6c18b9fc71152725022 implements a test for this.
datalog_2018_04_15_13_27_44_CEST_.h5
.sort_buffer_length
of 100.