Open XuanZhou001 opened 8 months ago
I think you might need to do this step on a node that has a GPU. Let me know if that works!
When I run mamba env create -f environment.yaml,An error will be reported here.
Could not solve for environment specs The following packages are incompatible ├─ tensorflow-base ==2.11.0 cuda112py310h52da4a5_0 is not installable because it requires │ └─ __cuda, which is missing on the system; ├─ tensorflow-estimator ==2.11.0 cuda112py310h37add04_0 is not installable because it requires │ └─ tensorflow-base 2.11.0 cuda112py310h52da4a5_0, which cannot be installed (as previously explained); └─ tensorflow ==2.11.0 cuda112py310he87a039_0 is not installable because it requires └─ __cuda, which is missing on the system.
I would be very happy if you could provide some help. Thank you!
Is cuda properly installed on your system? For example do the nvidia-smi
and nvcc --version
commands work?
Either way if this is giving problems I would recommend to update your nvidia drivers.
I have encountered this problem too, but I've successfully resolved it. In fact, I noticed an inconsistency in the environment.yaml file, as the required CUDA version was 11.8, as shown below:
- cuda-cudart=11.8.89=0
- cuda-cupti=11.8.87=0
- cuda-libraries=11.8.0=0
- cuda-nvrtc=11.8.89=0
- cuda-nvtx=11.8.86=0
- cuda-runtime=11.8.0=0
- cuda-version=11.8=h70ddcb2_3
- cudatoolkit=11.8.0=h4ba93d1_13
- cudnn=8.8.0.121=hcdd5f01_4
However, tensorFlow requires CUDA version 11.2, as indicated here:
- tensorflow=2.11.0=cuda112py310he87a039_0
- tensorflow-base=2.11.0=cuda112py310h52da4a5_0
- tensorflow-estimator=2.11.0=cuda112py310h37add04_0
Therefore, you could consider modifying it to:
- tensorflow=2.11.0
- tensorflow-base=2.11.0
- tensorflow-estimator=2.11.0
When I run mamba env create -f environment.yaml,An error will be reported here. nvidia/linux-64 ??.?MB @ ??.?MB/s 0 failed 6.0s predector/linux-64 ??.?MB @ ??.?MB/s 0 failed 6.0s pyg/linux-64 ??.?MB @ ??.?MB/s 0 failed 6.0s conda-forge/linux-64 ??.?MB @ ??.?MB/s 0 failed 6.0s
Traceback (most recent call last):
File "/gpfs/hpc/home/duanranhui/share_group_folder/ENTER/lib/python3.10/site-packages/conda/exception_handler.py", line 17, in __call__
return func(*args, **kwargs)
File "/gpfs/hpc/home/duanranhui/share_group_folder/ENTER/lib/python3.10/site-packages/conda_env/cli/main.py", line 56, in do_call
exit_code = getattr(module, func_name)(arguments, parser)
File "/gpfs/hpc/home/duanranhui/share_group_folder/ENTER/lib/python3.10/site-packages/conda/notices/core.py", line 124, in wrapper
return func(*args, **kwargs)
File "/gpfs/hpc/home/duanranhui/share_group_folder/ENTER/lib/python3.10/site-packages/conda_env/cli/main_create.py", line 164, in execute
result[installer_type] = installer.install(
File "/gpfs/hpc/home/duanranhui/share_group_folder/ENTER/lib/python3.10/site-packages/mamba/mamba_env.py", line 54, in mamba_install
index = load_channels(pool, channel_urls, repos, prepend=False)
File "/gpfs/hpc/home/duanranhui/share_group_folder/ENTER/lib/python3.10/site-packages/mamba/utils.py", line 126, in load_channels
index = get_index(
File "/gpfs/hpc/home/duanranhui/share_group_folder/ENTER/lib/python3.10/site-packages/mamba/utils.py", line 107, in get_index
is_downloaded = dlist.download(api.MAMBA_DOWNLOAD_FAILFAST)
RuntimeError: Download error (6) Couldn't resolve host name [https://conda.anaconda.org/bioconda/noarch/repodata.json]
Could not resolve host: conda.anaconda.org
$ /gpfs/hpc/home/duanranhui/share_group_folder/ENTER/bin/mamba create -f environment.yaml
environment variables:
CIO_TEST=
active environment : base
active env location : /gpfs/hpc/home/duanranhui/share_group_folder/ENTER
shell level : 1
user config file : /gpfs/hpc/home/duanranhui/ailiqiang/.condarc
populated config files : /gpfs/hpc/home/duanranhui/share_group_folder/ENTER/.condarc conda version : 23.11.0 conda-build version : not installed python version : 3.10.13.final.0 solver : libmamba (default) virtual packages : archspec=1=cascadelake conda=23.11.0=0 cuda=12.2=0 glibc=2.17=0 linux=3.10.0=0 unix=0=0 base environment : /gpfs/hpc/home/duanranhui/share_group_folder/ENTER (writable) conda av data dir : /gpfs/hpc/home/duanranhui/share_group_folder/ENTER/etc/conda conda av metadata url : None channel URLs : https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch package cache : /gpfs/hpc/home/duanranhui/share_group_folder/ENTER/pkgs /gpfs/hpc/home/duanranhui/ailiqiang/.conda/pkgs envs directories : /gpfs/hpc/home/duanranhui/share_group_folder/ENTER/envs /gpfs/hpc/home/duanranhui/ailiqiang/.conda/envs platform : linux-64 user-agent : conda/23.11.0 requests/2.31.0 CPython/3.10.13 Linux/3.10.0-1160.el7.x86_64 centos/7.9.2009 glibc/2.17 solver/libmamba conda-libmamba-solver/23.12.0 libmambapy/1.5.5 UID:GID : 32128:42040 netrc file : None offline mode : False
An unexpected error has occurred. Conda has prepared the above report. If you suspect this error is being caused by a malfunctioning plugin, consider using the --no-plugins option to turn off plugins.
Example: conda --no-plugins install
Alternatively, you can set the CONDA_NO_PLUGINS environment variable on the command line to run the command without plugins enabled.
Example: CONDA_NO_PLUGINS=true conda install
Because your server session likely doesn't give you GPU access, you're going to face issues getting the CUDA requirements installed with conda. To get around this, you can make the conda/mamba environment ignore this issue by typing:
CONDA_OVERRIDE_CUDA="11.2" conda env create -f environment.yaml
This has been suggested by the conda-forge team
There's no need to modify the script after this.
When I run mamba env create -f environment.yaml,An error will be reported here.
I would be very happy if you could provide some help. Thank you!