For future considerations we can add the ability to do distributed loading so you can use the dask-dashboard which is helpful for debugging these types of things. Is this a typical dataset size for you? Are you interested in taking larger datasets? If so then we can spend a bit of time streamlining/optimizing this workflow.
For future considerations we can add the ability to do distributed loading so you can use the dask-dashboard which is helpful for debugging these types of things. Is this a typical dataset size for you? Are you interested in taking larger datasets? If so then we can spend a bit of time streamlining/optimizing this workflow.
Originally posted by @CSSFrancis in https://github.com/hyperspy/rosettasciio/issues/329#issuecomment-2444375534
It should be very similar to PRs https://github.com/hyperspy/rosettasciio/pull/162 and https://github.com/hyperspy/rosettasciio/pull/267 since the
rippleusesmemmap.