search

xpsi-group / xpsi

X-PSI: X-ray Pulse Simulation and Inference
Other
34 stars 20 forks source link

The import order of pymultinest and numpy affects the seed-fixed results in certain conda environments #276

Open thjsal opened 1 year ago

thjsal commented 1 year ago

In both Python2 and Python3 versions of X-PSI, there seems to be a dependence of the exact MultiNest output on the order of pymultinestand numpy imports, if using a certain conda environment, even if fixing both MultiNest and NumPy seeds. Attached are the python2 conda environments, where the effect is seen and where it is not (at least during a short test run with an example script similar to that in examples/examples_fast/Modules/main.py): py2_environment_yes_effect.txt py2_environment_no_effect.txt

Pymultinest version is 2.11 in both (from commit 60c3490c7aefe126d8506e2dc025a0cc7f193512), and MultiNest version 3.12. Also tested that forcing the GCC compiler version (for X-PSI) to be the same for both does not change the results.

When the import order matters, it seems that all samples and their likelihoods are identical between the 2 cases, until a completely different parameter vector is given by MultiNest after hundreds of samples have been sampled.

thjsal commented 1 month ago

I am not sure if we need to keep this issue open or just close it. I don't know how to try to solve this, and whether it is actually a problem or not. Maybe fixing the seed does not simply guarantee the same end-result if changing even a small "supposedly irrelevant" thing, such as the import order of the modules (maybe the exact functioning of the random number generation depends somehow on this import order).