matthewfeickert / distributed-inference-with-pyhf-and-funcX

Example code for vCHEP 2021 paper "Distributed statistical inference with pyhf enabled through funcX"
MIT License
0 stars 2 forks source link

feat: Add environment lock file compiled with pip-tools #27

Closed matthewfeickert closed 2 years ago

matthewfeickert commented 2 years ago

Resolves #25

In an effort to produce more reproducible computing environments, which also are statically specified down to the hash level, while still only needing to control a small number of high level packages, use pip-tools's pip-compile to compile the dependencies of the packages specified in requirements.txt into a hash level requirements.lock lock file. Additionally, follow Brett Cannon's pip-secure-install recommendations to install exactly the same files that are specified in the lock file, for additional security and reproducibility.

To simplify things further, combine core-requirements.txt and jax-requirements.txt into a single requirements.txt.

Squash and merge commit message

* Add the 'minuit' pyhf extra to add iminuit to environment.
* Add compile_dependencies.sh that uses pip-compile to compile a hash level
requirements.lock lock file.
* Add requirements.lock lock file.
* Use pip-secure-install recommendations to install the environment from the
requirements.lock lock file inside the Docker image.
   - c.f. https://github.com/brettcannon/pip-secure-install
* Install the environment from requirements.lock in expanse-environment.yml
matthewfeickert commented 2 years ago

@BenGalewsky basically nothing should change here when it comes to building the Docker image, but let me know if you have any questions on all of this. :+1:

matthewfeickert commented 2 years ago

I'm going to merge this in as I think this should be uncontroversial, but I'm happy to discuss anything and to revert as needed. :+1: