search

pace-neutrons / Horace

Horace is a suite of programs for the visualization and analysis of large datasets from time-of-flight neutron inelastic scattering spectrometers.
https://pace-neutrons.github.io/Horace/stable/
GNU General Public License v3.0
8 stars 5 forks source link

Investigate standardising `distribute` #1585

Open oerc0122 opened 7 months ago

oerc0122 commented 7 months ago

Distribute currently uses its own implementation of split_vector_fixed_sum to account for the additional metadata, however, it may be possible to replace this, though it may require some re-working of the parallel merge procedures too.

Places which may be affected: ./horace_core/sqw/@DnDBase/distribute.m ./horace_core/sqw/@sqw/distribute.m ./horace_core/sqw/PixelData/@PixelDataMemory/distribute.m ./horace_core/sqw/PixelData/@PixelDataFileBacked/distribute.m ./herbert_core/utilities/classes/@IX_dataset/distribute.m ./herbert_core/utilities/misc/distribute.m ./herbert_core/classes/MPIFramework/parallel_call/ParallelSQWEval.m ./herbert_core/applications/multifit/MFParallel_Job.m

abuts commented 7 months ago

What I've seen in the code was reminiscent of selection between split_vector_fixed_sum and split_vector_max_sum || split_vector_max_sum_or_numel. I think thes covers all reasonable cases but may be something more is there too and another procedure should be introduced. If it is the case, it should be separately tested procedure with clear scope.