Closed matteoguarrera closed 1 month ago
yeah, agreed... I just use requirements.txt
and a venv
on NERSC to avoid conda issues on jobs. I think we can just source requirements.txt into the yaml file. Can you implement that? maybe 2 different reqs, one for gpu and one for cpu?
I actually create a conda environment using yaml on NERSC, so actually there is no need to keep requirements.txt
.
You can just conda env create -f env.yml
or conda env create -f env_gpu.yml
.
Or are you suggesting doing something like this?
name: my_environment
channels:
- conda-forge
dependencies:
- python=3.8
- pip
- pip:
-r requirements_cpu.txt # Replace with requirements_gpu.txt for GPU
yeah I mean the latter bc I still use reqs.txt
:P
But also because it is compatible with both workflows while the conda yaml doesn't support venv
.
we should use the same requirements between yaml and requirements.txt I noticed also that jax 0.4.20 raise some deprecation error. So I would suggest to push everything to jax 0.4.26. BTW I am also using NERSC