Open hippke opened 2 years ago
Do other stepsamplers work, such as SliceSampler (e.g., with generate_direction=generate_cube_oriented_direction, as CubeSliceSampler does)?
I think the AHARMSampler is still in development. It is intended to provide superior vectorization.
Thank you for the quick reply! SliceSampler
with generate_direction=generate_cube_oriented_direction
works, but is very slow (~50x fewer evals/sec compared to ReactiveNestedSampler).
My problem requires ~2e8 evals until convergence with the StepSampler. I can calculate 3e5 models per second on my GPU, so that convergence would be possible in 10 minutes. But the high GPU speed is only possible if I calculate the model in batches of >1e4 (i.e. vectorized). I struggle to feed these high numbers of models into UltraNest.
in ReactiveNestedSampler, you can control the batch size minimum and maximum with ndraw_min
and ndraw_max
, so I guess you could crank up ndraw_min
to 10000. This will give you extra evaluations, but if your GPU evals are cheap, it may effectively be faster.
SliceSampler currently does evolve one walker with hit-and-run Monte Carlo or slice sampling, always defining a direction, finding a bracket and then iteratively sampling from this slice and shrinking it until acceptance. This is repeated nsteps times. Since there is a dependence on the previous result, one cannot simply parallelise this for a single walker.
To truly take advantage of GPU parallelism, a sampler should:
If you are motivated, maybe you can implement something to this effect?
Description
I would like to run a vectorized stepsampler as described in issue 14. I can run the vectorized ReactiveNestedSampler correctly. For my problem, however, a stepsampler is much better suited. The vectorized stepsampler crashes with an error directly after launch.
What I Did
I run this code:
without the line
sampler.stepsampler = ultranest.stepsampler.AHARMSampler(nsteps=400)
it runs correctly. With this line it crashes: