Investigate python performance - Xana multi-tau correlator code

[RobertRosca]

Some benchmarking showed that SAXS and XPCS functions took up the majority of the code processing time, the profiling results can be seen here:

low, 10 trains, 2 jobs, 2 procs:

• SAXS: https://robertrosca.github.io/midtools-logs/01-low-use/dask-profile-SAXS.html
• XPCS: https://robertrosca.github.io/midtools-logs/01-low-use/dask-profile-XPCS.html med 50 trains, 4 jobs, 4 procs:
• SAXS: https://robertrosca.github.io/midtools-logs/02-med-use/dask-profile-SAXS.html
• XPCS: https://robertrosca.github.io/midtools-logs/02-med-use/dask-profile-XPCS.html high 100 trains, 8 jobs, 8 procs:
• SAXS: https://robertrosca.github.io/midtools-logs/03-high-use/dask-profile-SAXS.html
• XPCS: https://robertrosca.github.io/midtools-logs/03-high-use/dask-profile-XPCS.html

SAXS was largely bottlenecked by pyFAI, and there isn't too much we can do for that (apart from maybe looking a bit into ways to avoid potentially unneeded repeated calls to pyFAI?), but the XPCS functions come out of Xana, which we can more easily work on.

The main section of code is from Xana/XpcsAna/mp_corr3_err.py, finding a more well-optimised multi-tau autocorrelation function could potentially lead to a nice speedup in execution time.

[reiserm]

I agree that we cannot do too much about the performance of pyFAI. The difference between both cases (XPCS with Xana and SAXS with pyFai) is that XPCS is (currently) applied on a intra train level. Meaning that the correlation function is calculated for all images within a bunch-train. SAXS is applied per pulse where the train and pulse axes are stacked to the train_pulse MultiIndex with xarray. Maybe there is something we can do by changing the chunk size.

[reiserm]

We could actually try one more thing when we investigate the performance. In the setup-file (or config-file) we can actually switch off the multi-tau correlator (which should skip calling mp_corr3_err.py. Try to add

xpcs_opt:
    use_multitau: False

In the setupfile.