search

StanJulia / CmdStan.jl

CmdStan.jl v6 provides an alternative, older Julia wrapper to Stan's `cmdstan` executable. CmdStan will be deprecated in 2022.
MIT License
30 stars 12 forks source link

Restrict number of used cpu-cores #116

Closed pkroenert closed 2 years ago

pkroenert commented 2 years ago

Hi,

is it possible to restrict the number of used cpu-cores for parallel processing like it is possible with RSTAN?

See for example: https://github.com/stan-dev/rstan/wiki/RStan-Getting-Started' and within there the option: options(mc.cores = 80)

Thank you very much for your help!

goedman commented 2 years ago

Hi @PToTheK , I believe if you start Julia with julia -p auto it will create 4 threads on the Julia side (if you run 4 chains). There may be some additional threads if you compiled cmdstan multi-threaded.

If you start with julia -p 2 and look in the e.g. bernoulli_run.log you can see that when cmdstan is pulling draws it seems to use only 2 threads (instead of 4 earlier).

That is the (course) level of granularity control possible. I must admit, I wrote this Julia code over 10 years ago and haven't changed it much since then while a lot of work had been done on pmap and threads.

pkroenert commented 2 years ago

Thank you very much for the fast answer! I would like to ask you a followup question.

Does your code run each chain on a different core or thread?

Some context: I am analysing a multimodal model. Therefore I want to use 100 chains, wich should be evaluated in parallel on a condor-cluster.

goedman commented 2 years ago

HI @PToTheK Yes, each chain is a thread and allocated to cores as available.

Over time I've helped out in a few cluster setups. Best thing is to have a small test framework to check when your cluster needs specific things.

pkroenert commented 2 years ago

Hi @goedman, thank you very much once again for your quick and precise answer. This answer helps a lot!

Thank you very much for this great interface to STAN!