Using a number of cores larger than the number of chains

[JulioHC00]

Describe the bug

Not sure if this is a bug but it's something that's confusing me. The questions is, can we do multiprocessing with a number of cores larger than the number of chains? That is, when we do pm.sample, can you set cores to a number larger than chains? If so, I may be having issues because I can assign any number of cores to 2 chains (for example, 40 cores to 2 chains so each should get 20 cores) yet only 2 cores will ever be busy. So it appears as if there's only one core being assigned per chain. If setting cores>chains doesn't work, how can we make use of a system with a large number of cores to speed up calculations?

Your setup (please complete the following information):

• Version of exoplanet: 0.5.3
• Operating system: Ubuntu 22.04.1 LTS
• Python version & installation method (pip, conda, etc.): 3.10.6, pip

Additional context Same problem happens both in my personal laptop and in a server.

[dfm]

Hi - This isn't really an exoplanet issue, all of the sampling is done by PyMC and that package only supports parallelizing across chains. You won't get any benefit from using multiple cores per chain because none of the calculations are parallelized. I'm going to close this as a "won't fix" because that's just how PyMC works!

[JulioHC00]

Ah I see! So there's no way to make the sampling go faster than a single core per chain right?

[dfm]

Unfortunately there isn't! Some elements of the model (e.g. linear algebra) may use multiple threads if you have an accelerated linear algebra library (regardless of the setting for cores), but unfortunately that's it. Sorry!!