search

HEXRD / hexrd

A cross-platform, open-source library for the analysis of X-ray diffraction data.
Other
56 stars 25 forks source link

Lazily load the data in the frame-cache format #710

Closed psavery closed 1 month ago

psavery commented 2 months ago

Previously, just instantiating the imageseries would cause the data to be loaded. For some of the larger datasets from eiger, this could use a lot of RAM and time.

However, in some cases, we want to instantiate an imageseries just as a convenient (and somewhat standardized) way to look at things such as metadata (such as omegas) and other information without actually needing to load the data. This is actually something we do in the GUI to check the omegas (we do it for all image series types, in fact, not just frame-cache).

This commit changes the class to wait to load the data until it is actually requested. This makes loading it in the Simple Image Series loader in the GUI (which is done to check metadata) significantly faster.

This class still loads in all the frames the first time any frame is requested. Another potentially useful change would be to just load in whatever frame was requested and cache that frame in a dict (to be returned next time). However, maybe that could be done in the future if we encounter a more relevant use case for it.

psavery commented 2 months ago

@donald-e-boyce, can you review this one?

codecov[bot] commented 2 months ago

Codecov Report

Attention: Patch coverage is 94.44444% with 2 lines in your changes missing coverage. Please review.

Project coverage is 33.53%. Comparing base (433a0d5) to head (e9017ee).

Files Patch % Lines
hexrd/imageseries/load/framecache.py 94.44% 2 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #710 +/- ## ========================================== + Coverage 33.49% 33.53% +0.04% ========================================== Files 129 129 Lines 21237 21241 +4 ========================================== + Hits 7113 7123 +10 + Misses 14124 14118 -6 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

donald-e-boyce commented 1 month ago

Everything looks good, and my test problems work fine.

psavery commented 1 month ago

CI already passed except for the Mac test which was fixed in #713