OpenCOMPES / sed

Single Event Data Frame Processor: Backend to handle photoelectron resolved datastreams
https://opencompes.github.io/sed/
MIT License
3 stars 1 forks source link

faster version of per_file channels #431

Closed rettigl closed 2 weeks ago

rettigl commented 2 weeks ago

This PR implements an alternative method to generate the per_file columns, which is much more performant. Additionally, the function to add offsets is modified for a moderate speedup (removing the map_partitions call, and rather directly apply to the df columns)

coveralls commented 2 weeks ago

Pull Request Test Coverage Report for Build 9628505451

Details


Changes Missing Coverage Covered Lines Changed/Added Lines %
sed/loader/mpes/loader.py 33 39 84.62%
<!-- Total: 33 39 84.62% -->
Files with Coverage Reduction New Missed Lines %
sed/loader/mpes/loader.py 2 86.81%
<!-- Total: 2 -->
Totals Coverage Status
Change from base Build 9588080384: 0.008%
Covered Lines: 6500
Relevant Lines: 7065

💛 - Coveralls
coveralls commented 2 weeks ago

Pull Request Test Coverage Report for Build 9628850484

Details


Changes Missing Coverage Covered Lines Changed/Added Lines %
sed/loader/mpes/loader.py 33 39 84.62%
<!-- Total: 42 48 87.5% -->
Files with Coverage Reduction New Missed Lines %
sed/loader/mpes/loader.py 2 86.81%
<!-- Total: 2 -->
Totals Coverage Status
Change from base Build 9588080384: 0.003%
Covered Lines: 6496
Relevant Lines: 7061

💛 - Coveralls
rettigl commented 2 weeks ago

@zain-sohail This solution here passes the benchmarks, even with slight improvements on speed (maybe due to different worker specs), whereas the earlier code consistently is a factor 2-3 slower. I merge this here with the other branch, and then have you review it, if you want to see the changes, you can refer to this PR.