Describe the outcome that is desired.
Dependencies and version pinning requirements are currently spread across several files: the three mantid-developer environment files and the conda-build recipes. This duplication can easily lead to version pinnings becoming out of sync between developer environments and the conda-build recipes, meaning that we are testing against versions of dependencies that are different to the ones we are shipping against.
There are two separate but related outcomes that would be nice to resolve by consolidating requirements:
Specify our dependency versions in one place to be shared between the developer and conda-build environments.
Have a single mantid-developer environment definition to be shared between all OS. Currently there is no support for specifying that a particular package is only installed on a single OS (see here for a discussion on that).
Describe any solutions you are considering
Create a mantid-developerconda metapackage to be uploaded to the mantid channel.
Advantages:
can easily use the same pinnings as the other package recipes from conda_build_config.yaml
developers can easily create multiple development environments for testing changes to dependencies without having to rename the environment name in the current mantid-developer file.
Disadvantages:
would require an additional stage in the nightly pipeline, and at the start of all PR checks to re-build the mantid-developer conda package. In the nightly pipeline, we would deploy the package. In PR checks we would first check whether there are any changes to the recipe, and then build and install a local metapackage.
new metapackage would only be available after it has been deployed by the nightly pipeline, and wouldn't be directly in sync with main
might cause issues if there are differences between developer environments between main and release-next
Write a python script to parse a single yaml file cantaining OS selectors to generate an environment file and inject the version constraints from conda_build_config.yaml. This is the approach that was taken by the mantid imaging team (see here), where you run this command to create the developer environment:
python3 ./setup.py create_dev_env
Advantages:
developer env is always configured directly from source
does not require potentially lengthy conda-build step
no need to host a metapackage on the mantid channel
Disadvantages:
it's not a standard way of creating a conda environment
may be introducing new bugs
Additional context
This is essentially the same as this issue: https://github.com/mantidproject/mantid/issues/33442, but since the conda recipes have moved into the mantid repo, a lot of the information there is out of date.
Describe the outcome that is desired. Dependencies and version pinning requirements are currently spread across several files: the three
mantid-developerenvironment files and the conda-build recipes. This duplication can easily lead to version pinnings becoming out of sync between developer environments and the conda-build recipes, meaning that we are testing against versions of dependencies that are different to the ones we are shipping against.There are two separate but related outcomes that would be nice to resolve by consolidating requirements:
Describe any solutions you are considering
Create a
mantid-developerconda metapackage to be uploaded to themantidchannel.conda_build_config.yamlmainmainandrelease-nextWrite a python script to parse a single yaml file cantaining OS selectors to generate an environment file and inject the version constraints from
conda_build_config.yaml. This is the approach that was taken by the mantid imaging team (see here), where you run this command to create the developer environment:python3 ./setup.py create_dev_envAdditional context This is essentially the same as this issue: https://github.com/mantidproject/mantid/issues/33442, but since the conda recipes have moved into the mantid repo, a lot of the information there is out of date.