Closed zatkins2 closed 7 months ago
Hi Zach,
I just tested this by switching nthreads=1
to nthreads=0
in python/demos/sht_demo.py
, and I see a clear increase in speed. Do you observe the problem in a specific function?
Ah, and is this happening in a code that also uses OpenMP? If so, there might be some subtle interaction with OpenMP's thread pinning ...
I was using sht.experimental.adjoint_synthesis
, but I imagine this shares functionality for determining the number of threads with the rest of the module. Passing nthreads=x
, where x was anything >1, did properly use the requested number of threads, which is why I suspect the issue is related to determining the ducc0_max_threads
(though I can't imagine why it wouldn't work here, but would for fft
-- in that case, was using fft.r2c
).
Hm I suppose I have only tested in a code or shell that previously has imported pixell.curvedsky, which does indeed import its own compiled modules using OpenMP (e.g. cmisc
here). But the same goes for the fft.r2c
call.
I should add that I am testing without having set the OMP_NUM_THREADS
or DUCC0_NUM_THREADS
environment variables, though I think from looking at ducc0_max_threads
, that should still be fine when nthreads=0
.
Ah, I think I see the problem ... I might have been too clever for my own good in a place or two! Should be easy to fix; I'll let you know when I have an update.
Should now be fixed on HEAD - thanks for reporting this!
Nice!
BTW, the functions in ducc0.sht.experimental
are now also available in ducc0.sht
, since I think they have sufficiently matured by now.
Ah right, ok -- does that mean we should expect only sht
functions (not sht.experimental
functions) to be supported going forward?
The main way folks in ACT/SO are interfacing with ducc
is through pixell.curvedsky
which wraps the low-level ducc
functions. Currently the wrappers point to sht.experimental
, so we would need to update that and add a ducc>=0.33
dependency I guess.
There's no need to hurry ... I expect to keep the functions around in experimental
for at least another year.
I released verion 0.33 yesterday. Could you please verify that this version uses the correct number of threads?
It works as expected!
Hi Martin,
Apologies, I am long overdue on a response to my other question about lmax!
This is a bit odd, but it seems for me that ducc.sht.experimental functions can't seem to infer the number of available hardware cores if I pass nthreads=0. Passing nthreads=0 results in computation on 1 core. On the other hand, ducc.fft functions can do so.
I am using ducc 0.32 with a wheel built on my machine. The node has the following hardware: