search

astrostat / pylira

A Python package for Bayesian low-counts image reconstruction and analysis
9 stars 7 forks source link

Add random seed parameter #26

Closed adonath closed 2 years ago

adonath commented 2 years ago

This pull request adds a random_seed parameter to the bayes_image_analysis and image_analysis_R methods to allow for deterministic testing. I also introduced a LIRADeconvolver.random_state attribute which allow to handle the random seeds using np.random.RandomState. I adapted the tests to exact testing now, let's see if the CI passed and results are independent of the used platform.

adonath commented 2 years ago

Locally I now get reproducible results, however they seem to differ on the linux platform used for the CI tests. In general I think it's desirable to get cross-platform reproducible results. Right no I'm not 100% sure how to achieve this. I think it's related to #20, so we have to make sure to install the same R libraries on different platforms. Any other idea @infinitron how to get reproducible results across multiple platforms?

adonath commented 2 years ago

Just to add: locally I work with MacOS arm64 and R installed with brew.

infinitron commented 2 years ago

I think the problem could be with the way numpy generates random numbers (e.g., https://github.com/pytorch/pytorch/issues/5059). Can you try setting the random number directly instead of generating one from the random_state object?

adonath commented 2 years ago

@infinitron I added a unit test for the numpy random number generation and this passes: it yields the same results on my local as well as the CI platform (https://github.com/astrostat/pylira/runs/4241305956?check_suite_focus=true#step:6:86). So I think the issue must be on the C / R level most likely, because the R dependency differs between my local install (MacOS) and the one in the CI.

DougBurke commented 2 years ago

Just to note that you can find out packages are installed automatically via https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners at

In particular R is provided on recent ubuntu builds - e.g R 4.1.2 with Ubuntu 20.04 under the Tools section of https://github.com/actions/virtual-environments/blob/main/images/linux/Ubuntu2004-README.md

but it looks like they don't have a version for macOS (10.15 or 11) that I can see.