search

scipp / esspolarization

Polarization data reduction for the European Spallation Source
https://scipp.github.io/esspolarization/
BSD 3-Clause "New" or "Revised" License
0 stars 1 forks source link

Add lock around Mantid loading for 5x speedup #74

Closed SimonHeybrock closed 1 month ago

SimonHeybrock commented 1 month ago

See explanation in comment. I don't really know the reason for this. Setting Mantid's thread count to 1 had no effect, so I would suspect it has something to do with some file locks.

On my workstation computing the transmission fractions now takes 13 seconds instead of 90 seconds.

jokasimr commented 1 month ago

Should we do this anywhere else?

SimonHeybrock commented 1 month ago

Odd issue! But seems like a sensible solution.

Some questions just for curiosity: Are the dask threads loading the same file or are they all loading different files? Are the files they are trying to load hdf5 format or something else?

Different files. Yes, HDF5.

SimonHeybrock commented 1 month ago

Should we do this anywhere else?

Potentially in ESSsans (https://github.com/scipp/esssans/blob/main/src/ess/isissans/mantidio.py).