`docker build`: LibMambaUnsatisfiableError: Encountered problems while solving

josephrocca commented 8 months ago

Hello, I think there are some versions that need to be pinned to get this working according to the readme. The docker image doesn't build due to some libmamba errors. @ryanwhite04 ran into the same error today while trying it and IIRC he tried to play around with pinning various versions of various dependencies (based on the error logs - see below) to no avail.

Would you be able to share a working environment.yaml and environment_minimal.yaml with all versions pinned?

It looks like an exciting project based on your results - keen to try this out!

`docker build` logs for full requirements file:

Click for logs

``` docker build . -t josephrocca/tdmpc2:0.1.0 [+] Building 296.5s (11/12) => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 2.33kB 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/nvidia/cudagl:11.3.1-devel-ubuntu20.04 4.0s => [auth] nvidia/cudagl:pull token for registry-1.docker.io 0.0s => [1/7] FROM docker.io/nvidia/cudagl:11.3.1-devel-ubuntu20.04@sha256:46fb79db07c2773b241f6d851153b6043bffe08dd1efe2ae1daad 41.4s => => resolve docker.io/nvidia/cudagl:11.3.1-devel-ubuntu20.04@sha256:46fb79db07c2773b241f6d851153b6043bffe08dd1efe2ae1daadd 0.0s => => sha256:46fb79db07c2773b241f6d851153b6043bffe08dd1efe2ae1daadd3339929e45 3.47kB / 3.47kB 0.0s => => sha256:40ffec7023ca8113699b89ee237556cf8b3468155b5ee6907d307a7f6658bb0e 16.37kB / 16.37kB 0.0s => => sha256:086b79b77a0322664b136ca55b22dbacedb65c6e149dda481c9a47cebfa9ffd8 7.93MB / 7.93MB 2.3s => => sha256:d5fd17ec1767521cf97f61568096bfc9a7cd9c2d149576a7b43930c5a97062b0 28.57MB / 28.57MB 1.2s => => sha256:4698168f58887df18cd19bdfc48ced49675e18edf4f7f0f7492eab316f0f15f6 11.32MB / 11.32MB 0.4s => => sha256:86de3d5666669a8f6384a9289198994fcb47360f7dcd4bdb139895335e44fc9e 187B / 187B 1.0s => => sha256:30d00d5309895614303848d010aa542956b3e34a197169bb55a995c9156ee323 6.43kB / 6.43kB 1.7s => => extracting sha256:d5fd17ec1767521cf97f61568096bfc9a7cd9c2d149576a7b43930c5a97062b0 0.5s => => sha256:69a2bfee9a444226fb44897796f27839d955ff88a94b1e7f1fa52b3e0b263059 1.02GB / 1.02GB 22.0s => => sha256:381964195b8b6ac2a222260d039063574b186b59f1f7632f7193e4242f5491c4 62.04kB / 62.04kB 2.3s => => extracting sha256:086b79b77a0322664b136ca55b22dbacedb65c6e149dda481c9a47cebfa9ffd8 0.1s => => sha256:fe1468e51d2b1a58c3f50bf659bab5396b5d4e59d6f0533fdba00fb0862199a5 1.22GB / 1.22GB 26.0s => => sha256:e807ad87032f54918af3762e16215f5cb366274f499b80cfee46e29aa16734eb 84.89kB / 84.89kB 3.1s => => extracting sha256:4698168f58887df18cd19bdfc48ced49675e18edf4f7f0f7492eab316f0f15f6 0.1s => => extracting sha256:86de3d5666669a8f6384a9289198994fcb47360f7dcd4bdb139895335e44fc9e 0.0s => => extracting sha256:30d00d5309895614303848d010aa542956b3e34a197169bb55a995c9156ee323 0.0s => => sha256:535d29fae5aa35122502fa78d68a193781c17dcd35b4692302b1dcf52f880829 7.25MB / 7.25MB 10.4s => => sha256:bafc3ad22edcf47142dccc0f17daadaf06ce1374b76d32652bfc9b8a72f00ea9 196B / 196B 10.9s => => sha256:d47b7c086705bb4acda782763b3133ceee64f4aa41e3500c65f8511ac09b61c1 6.43kB / 6.43kB 11.4s => => sha256:cd1e76abf8701ee8bfbec294ab9da437d349e86213505955a5776d98d84941fa 92.89MB / 92.89MB 15.7s => => sha256:d091b2eb4fffc5832e0d11bd71b34e694cff19e2566d6b8697bedf14ec3f903e 300B / 300B 16.8s => => sha256:91420df31c156ccb75c83bda7d429194603dc372d48b0152a513238fab6d62ac 4.50MB / 4.50MB 18.9s => => extracting sha256:69a2bfee9a444226fb44897796f27839d955ff88a94b1e7f1fa52b3e0b263059 7.3s => => extracting sha256:381964195b8b6ac2a222260d039063574b186b59f1f7632f7193e4242f5491c4 0.0s => => extracting sha256:fe1468e51d2b1a58c3f50bf659bab5396b5d4e59d6f0533fdba00fb0862199a5 10.1s => => extracting sha256:e807ad87032f54918af3762e16215f5cb366274f499b80cfee46e29aa16734eb 0.0s => => extracting sha256:535d29fae5aa35122502fa78d68a193781c17dcd35b4692302b1dcf52f880829 0.1s => => extracting sha256:bafc3ad22edcf47142dccc0f17daadaf06ce1374b76d32652bfc9b8a72f00ea9 0.0s => => extracting sha256:d47b7c086705bb4acda782763b3133ceee64f4aa41e3500c65f8511ac09b61c1 0.0s => => extracting sha256:cd1e76abf8701ee8bfbec294ab9da437d349e86213505955a5776d98d84941fa 1.1s => => extracting sha256:d091b2eb4fffc5832e0d11bd71b34e694cff19e2566d6b8697bedf14ec3f903e 0.0s => => extracting sha256:91420df31c156ccb75c83bda7d429194603dc372d48b0152a513238fab6d62ac 0.1s => [internal] load build context 0.0s => => transferring context: 1.29kB 0.0s => [2/7] RUN apt-get -y update && apt-get install -y --no-install-recommends build-essential git nano rsync vim tree c 182.1s => [3/7] RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh && /bi 8.8s => [4/7] COPY nvidia_icd.json /usr/share/vulkan/icd.d/nvidia_icd.json 0.0s => [5/7] COPY environment.yaml /root 0.0s => ERROR [6/7] RUN conda env update -n base -f /root/environment.yaml && rm /root/environment.yaml && cd /root && 60.0s ------ > [6/7] RUN conda env update -n base -f /root/environment.yaml && rm /root/environment.yaml && cd /root && python -m mani_skill2.utils.download_asset all -y && conda clean -ya && pip cache purge: #0 0.976 Channels: #0 0.976 - pytorch-nightly #0 0.976 - nvidia #0 0.976 - conda-forge #0 0.976 - defaults #0 0.976 Platform: linux-64 #0 0.976 Collecting package metadata (repodata.json): ...working... done #0 27.25 Solving environment: ...working... failed #0 40.42 Channels: #0 40.42 - pytorch-nightly #0 40.42 - nvidia #0 40.42 - conda-forge #0 40.42 - defaults #0 40.42 Platform: linux-64 #0 40.42 Collecting package metadata (repodata.json): ...working... done #0 50.37 Solving environment: ...working... failed #0 59.86 #0 59.86 LibMambaUnsatisfiableError: Encountered problems while solving: #0 59.86 - package conda-23.11.0-py311h06a4308_0 requires python >=3.11,<3.12.0a0, but none of the providers can be installed #0 59.86 #0 59.86 Could not solve for environment specs #0 59.86 The following packages are incompatible #0 59.86 ├─ conda 23.11.0 is installable with the potential options #0 59.86 │ ├─ conda 23.11.0 would require #0 59.86 │ │ └─ python >=3.11,<3.12.0a0 , which can be installed; #0 59.86 │ ├─ conda 23.11.0 would require #0 59.86 │ │ ├─ python >=3.10,<3.11.0a0 , which can be installed; #0 59.86 │ │ └─ python_abi 3.10.* *_cp310 with the potential options #0 59.86 │ │ ├─ python_abi 3.10, which can be installed; #0 59.86 │ │ └─ python_abi 3.10 would require #0 59.86 │ │ └─ python 3.10.* *_cpython, which can be installed; #0 59.86 │ ├─ conda 23.11.0 would require #0 59.86 │ │ ├─ python >=3.11,<3.12.0a0 , which can be installed; #0 59.86 │ │ └─ python_abi 3.11.* *_cp311 with the potential options #0 59.86 │ │ ├─ python_abi 3.11, which can be installed; #0 59.86 │ │ └─ python_abi 3.11 would require #0 59.86 │ │ └─ python 3.11.* *_cpython, which can be installed; #0 59.86 │ ├─ conda 23.11.0 would require #0 59.86 │ │ └─ python_abi 3.12.* *_cp312, which requires #0 59.86 │ │ └─ python 3.12.* *_cpython, which can be installed; #0 59.86 │ ├─ conda 23.11.0 would require #0 59.86 │ │ ├─ python >=3.8,<3.9.0a0 , which can be installed; #0 59.86 │ │ └─ python_abi 3.8.* *_cp38 with the potential options #0 59.86 │ │ ├─ python_abi 3.8, which can be installed; #0 59.86 │ │ └─ python_abi 3.8 would require #0 59.86 │ │ └─ python 3.8.* *_cpython, which can be installed; #0 59.86 │ ├─ conda 23.11.0 would require #0 59.86 │ │ └─ pypy3.9 >=7.3.13 , which requires #0 59.86 │ │ └─ python 3.9.* *_73_pypy, which can be installed; #0 59.86 │ ├─ conda 23.11.0 would require #0 59.86 │ │ └─ conda-libmamba-solver >=23.11.0 with the potential options #0 59.86 │ │ ├─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 59.86 │ │ │ └─ libmambapy >=1.5.3 with the potential options #0 59.86 │ │ │ ├─ libmambapy 1.5.3 would require #0 59.86 │ │ │ │ └─ openssl >=3.0.11,<4.0a0 , which can be installed; #0 59.86 │ │ │ ├─ libmambapy [1.5.3|1.5.6] would require #0 59.86 │ │ │ │ └─ python >=3.11,<3.12.0a0 , which can be installed; #0 59.86 │ │ │ ├─ libmambapy [1.5.3|1.5.4|1.5.5|1.5.6] would require #0 59.86 │ │ │ │ ├─ python >=3.10,<3.11.0a0 , which can be installed; #0 59.86 │ │ │ │ └─ python_abi 3.10.* *_cp310 with the potential options #0 59.86 │ │ │ │ ├─ python_abi 3.10, which can be installed; #0 59.86 │ │ │ │ └─ python_abi 3.10, which can be installed (as previously explained); #0 59.86 │ │ │ ├─ libmambapy [1.5.3|1.5.4|1.5.5|1.5.6] would require #0 59.86 │ │ │ │ ├─ python >=3.11,<3.12.0a0 , which can be installed; #0 59.86 │ │ │ │ └─ python_abi 3.11.* *_cp311 with the potential options #0 59.86 │ │ │ │ ├─ python_abi 3.11, which can be installed; #0 59.86 │ │ │ │ └─ python_abi 3.11, which can be installed (as previously explained); #0 59.86 │ │ │ ├─ libmambapy [1.5.3|1.5.4|1.5.5|1.5.6] would require #0 59.86 │ │ │ │ └─ python_abi 3.12.* *_cp312, which can be installed (as previously explained); #0 59.86 │ │ │ ├─ libmambapy [1.5.3|1.5.4|1.5.5|1.5.6] would require #0 59.86 │ │ │ │ ├─ python >=3.8,<3.9.0a0 , which can be installed; #0 59.86 │ │ │ │ └─ python_abi 3.8.* *_cp38 with the potential options #0 59.86 │ │ │ │ ├─ python_abi 3.8, which can be installed; #0 59.86 │ │ │ │ └─ python_abi 3.8, which can be installed (as previously explained); #0 59.86 │ │ │ ├─ libmambapy 1.5.3 would require #0 59.86 │ │ │ │ └─ openssl >=3.1.4,<4.0a0 , which can be installed; #0 59.86 │ │ │ ├─ libmambapy [1.5.3|1.5.4|1.5.5|1.5.6] would require #0 59.86 │ │ │ │ └─ pypy3.9 >=7.3.13 , which can be installed (as previously explained); #0 59.86 │ │ │ ├─ libmambapy [1.5.4|1.5.5|1.5.6] would require #0 59.86 │ │ │ │ └─ openssl >=3.2.0,<4.0a0 , which can be installed; #0 59.86 │ │ │ ├─ libmambapy [1.5.3|1.5.6] would require #0 59.86 │ │ │ │ └─ python >=3.10,<3.11.0a0 , which can be installed; #0 59.86 │ │ │ ├─ libmambapy 1.5.3 would require #0 59.86 │ │ │ │ └─ python >=3.12,<3.13.0a0 , which can be installed; #0 59.86 │ │ │ ├─ libmambapy [1.5.3|1.5.6] would require #0 59.86 │ │ │ │ └─ python >=3.8,<3.9.0a0 , which can be installed; #0 59.86 │ │ │ └─ libmambapy 1.5.6 would require #0 59.86 │ │ │ └─ openssl >=3.0.12,<4.0a0 , which can be installed; #0 59.86 │ │ ├─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 59.86 │ │ │ └─ python >=3.10,<3.11.0a0 , which can be installed; #0 59.86 │ │ ├─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 59.86 │ │ │ └─ python >=3.11,<3.12.0a0 , which can be installed; #0 59.86 │ │ └─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 59.86 │ │ └─ python >=3.8,<3.9.0a0 , which can be installed; #0 59.86 │ ├─ conda 23.11.0 would require #0 59.86 │ │ └─ python >=3.10,<3.11.0a0 , which can be installed; #0 59.86 │ └─ conda 23.11.0 would require #0 59.86 │ └─ python >=3.8,<3.9.0a0 , which can be installed; #0 59.86 └─ python 3.9.0** is not installable because there are no viable options #0 59.86 ├─ python 3.9.0 would require #0 59.86 │ └─ openssl >=1.1.1h,<1.1.2a , which conflicts with any installable versions previously reported; #0 59.86 └─ python 3.9.0 would require #0 59.86 ├─ openssl >=1.1.1h,<1.1.2a , which conflicts with any installable versions previously reported; #0 59.86 └─ python_abi 3.9.* *_cp39, which conflicts with any installable versions previously reported. #0 59.86 ------ Dockerfile:39 -------------------- 38 | COPY environment.yaml /root 39 | >>> RUN conda env update -n base -f /root/environment.yaml && \ 40 | >>> rm /root/environment.yaml && \ 41 | >>> cd /root && \ 42 | >>> python -m mani_skill2.utils.download_asset all -y && \ 43 | >>> conda clean -ya && \ 44 | >>> pip cache purge 45 | -------------------- ERROR: failed to solve: process "/bin/bash -c conda env update -n base -f /root/environment.yaml && rm /root/environment.yaml && cd /root && python -m mani_skill2.utils.download_asset all -y && conda clean -ya && pip cache purge" did not complete successfully: exit code: 1 ```

`docker build` logs for minimal requirements file:

Click for logs

``` docker build . -t josephrocca/tdmpc2:0.1.0 [+] Building 64.8s (11/12) => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 2.35kB 0.0s => [internal] load metadata for docker.io/nvidia/cudagl:11.3.1-devel-ubuntu20.04 4.2s => [auth] nvidia/cudagl:pull token for registry-1.docker.io 0.0s => [1/7] FROM docker.io/nvidia/cudagl:11.3.1-devel-ubuntu20.04@sha256:46fb79db07c2773b241f6d851153b6043bffe08dd1efe2ae1daadd 0.0s => [internal] load build context 0.0s => => transferring context: 687B 0.0s => CACHED [2/7] RUN apt-get -y update && apt-get install -y --no-install-recommends build-essential git nano rsync vim t 0.0s => CACHED [3/7] RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh && 0.0s => CACHED [4/7] COPY nvidia_icd.json /usr/share/vulkan/icd.d/nvidia_icd.json 0.0s => [5/7] COPY environment_minimal.yaml /root 0.0s => ERROR [6/7] RUN conda env update -n base -f /root/environment_minimal.yaml && rm /root/environment_minimal.yaml && 60.5s ------ > [6/7] RUN conda env update -n base -f /root/environment_minimal.yaml && rm /root/environment_minimal.yaml && cd /root && python -m mani_skill2.utils.download_asset all -y && conda clean -ya && pip cache purge: #0 0.873 Channels: #0 0.873 - pytorch-nightly #0 0.873 - nvidia #0 0.873 - conda-forge #0 0.873 - defaults #0 0.873 Platform: linux-64 #0 0.873 Collecting package metadata (repodata.json): ...working... done #0 28.29 Solving environment: ...working... failed #0 41.70 Channels: #0 41.70 - pytorch-nightly #0 41.70 - nvidia #0 41.70 - conda-forge #0 41.70 - defaults #0 41.70 Platform: linux-64 #0 41.70 Collecting package metadata (repodata.json): ...working... done #0 51.52 Solving environment: ...working... failed #0 60.43 #0 60.43 LibMambaUnsatisfiableError: Encountered problems while solving: #0 60.43 - package conda-23.11.0-py311h06a4308_0 requires python >=3.11,<3.12.0a0, but none of the providers can be installed #0 60.43 #0 60.43 Could not solve for environment specs #0 60.43 The following packages are incompatible #0 60.43 ├─ conda 23.11.0 is installable with the potential options #0 60.43 │ ├─ conda 23.11.0 would require #0 60.43 │ │ └─ python >=3.11,<3.12.0a0 , which can be installed; #0 60.43 │ ├─ conda 23.11.0 would require #0 60.43 │ │ ├─ python >=3.10,<3.11.0a0 , which can be installed; #0 60.43 │ │ └─ python_abi 3.10.* *_cp310 with the potential options #0 60.43 │ │ ├─ python_abi 3.10, which can be installed; #0 60.43 │ │ └─ python_abi 3.10 would require #0 60.43 │ │ └─ python 3.10.* *_cpython, which can be installed; #0 60.43 │ ├─ conda 23.11.0 would require #0 60.43 │ │ ├─ python >=3.11,<3.12.0a0 , which can be installed; #0 60.43 │ │ └─ python_abi 3.11.* *_cp311 with the potential options #0 60.43 │ │ ├─ python_abi 3.11, which can be installed; #0 60.43 │ │ └─ python_abi 3.11 would require #0 60.43 │ │ └─ python 3.11.* *_cpython, which can be installed; #0 60.43 │ ├─ conda 23.11.0 would require #0 60.43 │ │ └─ python_abi 3.12.* *_cp312, which requires #0 60.43 │ │ └─ python 3.12.* *_cpython, which can be installed; #0 60.43 │ ├─ conda 23.11.0 would require #0 60.43 │ │ ├─ python >=3.8,<3.9.0a0 , which can be installed; #0 60.43 │ │ └─ python_abi 3.8.* *_cp38 with the potential options #0 60.43 │ │ ├─ python_abi 3.8, which can be installed; #0 60.43 │ │ └─ python_abi 3.8 would require #0 60.43 │ │ └─ python 3.8.* *_cpython, which can be installed; #0 60.43 │ ├─ conda 23.11.0 would require #0 60.43 │ │ └─ pypy3.9 >=7.3.13 , which requires #0 60.43 │ │ └─ python 3.9.* *_73_pypy, which can be installed; #0 60.43 │ ├─ conda 23.11.0 would require #0 60.43 │ │ └─ conda-libmamba-solver >=23.11.0 with the potential options #0 60.43 │ │ ├─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 60.43 │ │ │ └─ libmambapy >=1.5.3 with the potential options #0 60.43 │ │ │ ├─ libmambapy 1.5.3 would require #0 60.43 │ │ │ │ └─ openssl >=3.0.11,<4.0a0 , which can be installed; #0 60.43 │ │ │ ├─ libmambapy [1.5.3|1.5.6] would require #0 60.43 │ │ │ │ └─ python >=3.11,<3.12.0a0 , which can be installed; #0 60.43 │ │ │ ├─ libmambapy [1.5.3|1.5.4|1.5.5|1.5.6] would require #0 60.43 │ │ │ │ ├─ python >=3.10,<3.11.0a0 , which can be installed; #0 60.43 │ │ │ │ └─ python_abi 3.10.* *_cp310 with the potential options #0 60.43 │ │ │ │ ├─ python_abi 3.10, which can be installed; #0 60.43 │ │ │ │ └─ python_abi 3.10, which can be installed (as previously explained); #0 60.43 │ │ │ ├─ libmambapy [1.5.3|1.5.4|1.5.5|1.5.6] would require #0 60.43 │ │ │ │ ├─ python >=3.11,<3.12.0a0 , which can be installed; #0 60.43 │ │ │ │ └─ python_abi 3.11.* *_cp311 with the potential options #0 60.43 │ │ │ │ ├─ python_abi 3.11, which can be installed; #0 60.43 │ │ │ │ └─ python_abi 3.11, which can be installed (as previously explained); #0 60.43 │ │ │ ├─ libmambapy [1.5.3|1.5.4|1.5.5|1.5.6] would require #0 60.43 │ │ │ │ └─ python_abi 3.12.* *_cp312, which can be installed (as previously explained); #0 60.43 │ │ │ ├─ libmambapy [1.5.3|1.5.4|1.5.5|1.5.6] would require #0 60.43 │ │ │ │ ├─ python >=3.8,<3.9.0a0 , which can be installed; #0 60.43 │ │ │ │ └─ python_abi 3.8.* *_cp38 with the potential options #0 60.43 │ │ │ │ ├─ python_abi 3.8, which can be installed; #0 60.43 │ │ │ │ └─ python_abi 3.8, which can be installed (as previously explained); #0 60.43 │ │ │ ├─ libmambapy 1.5.3 would require #0 60.43 │ │ │ │ └─ openssl >=3.1.4,<4.0a0 , which can be installed; #0 60.43 │ │ │ ├─ libmambapy [1.5.3|1.5.4|1.5.5|1.5.6] would require #0 60.43 │ │ │ │ └─ pypy3.9 >=7.3.13 , which can be installed (as previously explained); #0 60.43 │ │ │ ├─ libmambapy [1.5.4|1.5.5|1.5.6] would require #0 60.43 │ │ │ │ └─ openssl >=3.2.0,<4.0a0 , which can be installed; #0 60.43 │ │ │ ├─ libmambapy [1.5.3|1.5.6] would require #0 60.43 │ │ │ │ └─ python >=3.10,<3.11.0a0 , which can be installed; #0 60.43 │ │ │ ├─ libmambapy 1.5.3 would require #0 60.43 │ │ │ │ └─ python >=3.12,<3.13.0a0 , which can be installed; #0 60.43 │ │ │ ├─ libmambapy [1.5.3|1.5.6] would require #0 60.43 │ │ │ │ └─ python >=3.8,<3.9.0a0 , which can be installed; #0 60.43 │ │ │ └─ libmambapy 1.5.6 would require #0 60.43 │ │ │ └─ openssl >=3.0.12,<4.0a0 , which can be installed; #0 60.43 │ │ ├─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 60.43 │ │ │ └─ python >=3.10,<3.11.0a0 , which can be installed; #0 60.43 │ │ ├─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 60.43 │ │ │ └─ python >=3.11,<3.12.0a0 , which can be installed; #0 60.43 │ │ └─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 60.43 │ │ └─ python >=3.8,<3.9.0a0 , which can be installed; #0 60.43 │ ├─ conda 23.11.0 would require #0 60.43 │ │ └─ python >=3.10,<3.11.0a0 , which can be installed; #0 60.43 │ └─ conda 23.11.0 would require #0 60.43 │ └─ python >=3.8,<3.9.0a0 , which can be installed; #0 60.43 └─ python 3.9.0** is not installable because there are no viable options #0 60.43 ├─ python 3.9.0 would require #0 60.43 │ └─ openssl >=1.1.1h,<1.1.2a , which conflicts with any installable versions previously reported; #0 60.43 └─ python 3.9.0 would require #0 60.43 ├─ openssl >=1.1.1h,<1.1.2a , which conflicts with any installable versions previously reported; #0 60.43 └─ python_abi 3.9.* *_cp39, which conflicts with any installable versions previously reported. #0 60.43 ------ Dockerfile:39 -------------------- 38 | COPY environment_minimal.yaml /root 39 | >>> RUN conda env update -n base -f /root/environment_minimal.yaml && \ 40 | >>> rm /root/environment_minimal.yaml && \ 41 | >>> cd /root && \ 42 | >>> python -m mani_skill2.utils.download_asset all -y && \ 43 | >>> conda clean -ya && \ 44 | >>> pip cache purge 45 | -------------------- ERROR: failed to solve: process "/bin/bash -c conda env update -n base -f /root/environment_minimal.yaml && rm /root/environment_minimal.yaml && cd /root && python -m mani_skill2.utils.download_asset all -y && conda clean -ya && pip cache purge" did not complete successfully: exit code: 1 ```

nicklashansen commented 8 months ago

Thanks for reaching out! I'll be happy to help you resolve this issue.

It has been a little while since I last compiled a docker image using this script, but I built a fresh conda env using conda env create -f docker/environment_minimal.yaml just now using the latest commit. This is the exported env:

name: tdmpc2
channels:
  - pytorch-nightly
  - nvidia
  - conda-forge
  - defaults
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_gnu
  - aom=3.4.0=h27087fc_1
  - blas=1.0=mkl
  - brotli-python=1.1.0=py39h3d6467e_1
  - bzip2=1.0.8=hd590300_5
  - ca-certificates=2023.11.17=hbcca054_0
  - certifi=2023.11.17=pyhd8ed1ab_0
  - charset-normalizer=3.3.2=pyhd8ed1ab_0
  - cuda-cudart=12.1.105=0
  - cuda-cupti=12.1.105=0
  - cuda-libraries=12.1.0=0
  - cuda-nvrtc=12.1.105=0
  - cuda-nvtx=12.1.105=0
  - cuda-opencl=12.3.101=0
  - cuda-runtime=12.1.0=0
  - cudatoolkit=11.7.0=hd8887f6_10
  - expat=2.5.0=hcb278e6_1
  - filelock=3.13.1=pyhd8ed1ab_0
  - font-ttf-dejavu-sans-mono=2.37=hab24e00_0
  - font-ttf-inconsolata=3.000=h77eed37_0
  - font-ttf-source-code-pro=2.038=h77eed37_0
  - font-ttf-ubuntu=0.83=h77eed37_1
  - fontconfig=2.14.2=h14ed4e7_0
  - fonts-conda-ecosystem=1=0
  - fonts-conda-forge=1=0
  - freetype=2.12.1=h267a509_2
  - gettext=0.21.1=h27087fc_0
  - glew=2.1.0=h9c3ff4c_2
  - glib=2.68.4=h9c3ff4c_0
  - glib-tools=2.68.4=h9c3ff4c_0
  - gmp=6.3.0=h59595ed_0
  - gmpy2=2.1.2=py39h376b7d2_1
  - gnutls=3.7.6=hbf5b4be_4
  - icu=72.1=hcb278e6_0
  - idna=3.6=pyhd8ed1ab_0
  - intel-openmp=2022.1.0=h9e868ea_3769
  - jinja2=3.1.2=pyhd8ed1ab_1
  - lame=3.100=h166bdaf_1003
  - lcms2=2.16=hb7c19ff_0
  - ld_impl_linux-64=2.40=h41732ed_0
  - lerc=4.0.0=h27087fc_0
  - libblas=3.9.0=16_linux64_mkl
  - libcblas=3.9.0=16_linux64_mkl
  - libcublas=12.1.0.26=0
  - libcufft=11.0.2.4=0
  - libcufile=1.8.1.2=0
  - libcurand=10.3.4.107=0
  - libcusolver=11.4.4.55=0
  - libcusparse=12.0.2.55=0
  - libdeflate=1.19=hd590300_0
  - libdrm=2.4.114=h166bdaf_0
  - libexpat=2.5.0=hcb278e6_1
  - libffi=3.3=h58526e2_2
  - libgcc-ng=13.2.0=h807b86a_3
  - libglib=2.68.4=h3e27bee_0
  - libglu=9.0.0=hac7e632_1003
  - libgomp=13.2.0=h807b86a_3
  - libiconv=1.17=hd590300_2
  - libidn2=2.3.4=h166bdaf_0
  - libjpeg-turbo=3.0.0=hd590300_1
  - liblapack=3.9.0=16_linux64_mkl
  - libnpp=12.0.2.50=0
  - libnvjitlink=12.1.105=0
  - libnvjpeg=12.1.1.14=0
  - libpciaccess=0.17=h166bdaf_0
  - libpng=1.6.39=h753d276_0
  - libsqlite=3.44.2=h2797004_0
  - libstdcxx-ng=13.2.0=h7e041cc_3
  - libtasn1=4.19.0=h166bdaf_0
  - libtiff=4.6.0=ha9c0a0a_2
  - libunistring=0.9.10=h7f98852_0
  - libuuid=2.38.1=h0b41bf4_0
  - libva=2.20.0=hd590300_0
  - libvpx=1.11.0=h9c3ff4c_3
  - libwebp-base=1.3.2=hd590300_0
  - libxcb=1.15=h0b41bf4_0
  - libxml2=2.10.4=hfdac1af_0
  - libzlib=1.2.13=hd590300_5
  - llvm-openmp=15.0.7=h0cdce71_0
  - markupsafe=2.1.3=py39hd1e30aa_1
  - mkl=2022.1.0=hc2b9512_224
  - mpc=1.3.1=hfe3b2da_0
  - mpfr=4.2.1=h9458935_0
  - mpmath=1.3.0=pyhd8ed1ab_0
  - ncurses=6.4=h59595ed_2
  - nettle=3.8.1=hc379101_1
  - networkx=3.2.1=pyhd8ed1ab_0
  - numpy=1.26.3=py39h474f0d3_0
  - openh264=2.3.0=h27087fc_0
  - openjpeg=2.5.0=h488ebb8_3
  - openssl=1.1.1w=hd590300_0
  - p11-kit=0.23.21=hb875675_1
  - pcre=8.45=h9c3ff4c_0
  - pillow=10.2.0=py39had0adad_0
  - pip=21.0=pyhd8ed1ab_1
  - pthread-stubs=0.4=h36c2ea0_1001
  - pysocks=1.7.1=pyha2e5f31_6
  - python=3.9.0=hffdb5ce_5_cpython
  - python_abi=3.9=4_cp39
  - pytorch=2.3.0.dev20240110=py3.9_cuda12.1_cudnn8.9.2_0
  - pytorch-cuda=12.1=ha16c6d3_5
  - pytorch-mutex=1.0=cuda
  - pyyaml=6.0.1=py39hd1e30aa_1
  - readline=8.2=h8228510_1
  - requests=2.31.0=pyhd8ed1ab_0
  - sqlite=3.44.2=h2c6b66d_0
  - svt-av1=1.2.0=h27087fc_0
  - sympy=1.12=pypyh9d50eac_103
  - tk=8.6.13=noxft_h4845f30_101
  - torchtriton=2.2.0+e28a256d71=py39
  - torchvision=0.18.0.dev20240110=py39_cu121
  - typing_extensions=4.9.0=pyha770c72_0
  - urllib3=2.1.0=pyhd8ed1ab_0
  - wheel=0.42.0=pyhd8ed1ab_0
  - x264=1!164.3095=h166bdaf_2
  - x265=3.5=h924138e_3
  - xorg-fixesproto=5.0=h7f98852_1002
  - xorg-kbproto=1.0.7=h7f98852_1002
  - xorg-libx11=1.8.7=h8ee46fc_0
  - xorg-libxau=1.0.11=hd590300_0
  - xorg-libxdmcp=1.1.3=h7f98852_0
  - xorg-libxext=1.3.4=h0b41bf4_2
  - xorg-libxfixes=5.0.3=h7f98852_1004
  - xorg-xextproto=7.3.0=h0b41bf4_1003
  - xorg-xproto=7.0.31=h7f98852_1007
  - xz=5.2.6=h166bdaf_0
  - yaml=0.2.5=h7f98852_2
  - zlib=1.2.13=hd590300_5
  - zstd=1.5.5=hfc55251_0
  - pip:
      - absl-py==2.0.0
      - antlr4-python3-runtime==4.9.3
      - appdirs==1.4.4
      - click==8.1.7
      - cloudpickle==3.0.0
      - cython==0.29.37
      - decorator==4.4.2
      - dm-control==1.0.16
      - dm-env==1.6
      - dm-tree==0.1.8
      - docker-pycreds==0.4.0
      - etils==1.5.2
      - ffmpeg==1.4
      - fsspec==2023.12.2
      - gitdb==4.0.11
      - gitpython==3.1.41
      - glfw==2.6.4
      - gym==0.21.0
      - hydra-core==1.3.2
      - hydra-submitit-launcher==1.2.0
      - imageio==2.33.1
      - imageio-ffmpeg==0.4.9
      - importlib-resources==6.1.1
      - kornia==0.7.1
      - labmaze==1.0.6
      - lxml==5.1.0
      - moviepy==1.0.3
      - mujoco==3.1.1
      - omegaconf==2.3.0
      - packaging==23.2
      - pandas==2.1.4
      - patchelf==0.17.2.1
      - proglog==0.1.10
      - protobuf==4.25.2
      - psutil==5.9.7
      - pyopengl==3.1.7
      - pyparsing==3.1.1
      - python-dateutil==2.8.2
      - pytz==2023.3.post1
      - scipy==1.11.4
      - sentry-sdk==1.39.2
      - setproctitle==1.3.3
      - setuptools==65.5.0
      - six==1.16.0
      - smmap==5.0.1
      - submitit==1.5.1
      - tensordict-nightly==2024.1.10
      - termcolor==2.4.0
      - torchrl-nightly==2024.1.10
      - tqdm==4.66.1
      - tzdata==2023.4
      - wandb==0.16.2
      - zipp==3.17.0

I have checked that I am able to run python train.py (defaults to DMControl) with this env. Let me know if this environment file does not resolve your issue; in that case I might have to verify that the dockerfile still builds for me as well.

josephrocca commented 8 months ago

Thanks for your response! I've just tried this and I still get a similar LibMambaUnsatisfiableError error for some reason:

package libmambapy-1.5.3-py311h2dafd23_0 requires openssl >=3.0.11,<4.0a0, but none of the providers can be installed

I confirmed with @ryanwhite04 just now that he gets the same error.

For reference, I'm using Ubuntu/PopOS 22.04 and Ryan is using WSL within Windows 11.

Steps I took:

Copy the yaml you gave into environment_minimal.yaml
Ensure all instances of environment.yaml are replaced with environment_minimal.yaml in the Dockerfile
Run docker build . -t josephrocca/tdmpc2:0.1.0 from within the docker directory
Observe error logs attached below

I also tried git cloneing a fresh repo (since I see you made some changes today) and trying the above steps, but still got the same error.

(also, as a guess, I tried python=3.11.0, since the error logs below seem to suggest that might work, but it didn't)

Error logs

Click for full logs

``` docker build . -t josephrocca/tdmpc2:0.1.0 [+] Building 68.8s (11/12) => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 2.35kB 0.0s => [internal] load metadata for docker.io/nvidia/cudagl:11.3.1-devel-ubu 3.0s => [auth] nvidia/cudagl:pull token for registry-1.docker.io 0.0s => [internal] load build context 0.0s => => transferring context: 5.56kB 0.0s => [1/7] FROM docker.io/nvidia/cudagl:11.3.1-devel-ubuntu20.04@sha256:46 0.0s => CACHED [2/7] RUN apt-get -y update && apt-get install -y --no-ins 0.0s => CACHED [3/7] RUN wget --quiet https://repo.anaconda.com/miniconda/Min 0.0s => CACHED [4/7] COPY nvidia_icd.json /usr/share/vulkan/icd.d/nvidia_icd. 0.0s => [5/7] COPY environment_minimal.yaml /root 0.0s => ERROR [6/7] RUN conda env update -n base -f /root/environment_minima 65.7s ------ > [6/7] RUN conda env update -n base -f /root/environment_minimal.yaml && rm /root/environment_minimal.yaml && cd /root && python -m mani_skill2.utils.download_asset all -y && conda clean -ya && pip cache purge: #0 0.910 Channels: #0 0.910 - pytorch-nightly #0 0.910 - nvidia #0 0.910 - conda-forge #0 0.910 - defaults #0 0.910 Platform: linux-64 #0 0.910 Collecting package metadata (repodata.json): ...working... done #0 27.44 Solving environment: ...working... failed #0 45.77 Channels: #0 45.77 - pytorch-nightly #0 45.77 - nvidia #0 45.77 - conda-forge #0 45.77 - defaults #0 45.77 Platform: linux-64 #0 45.77 Collecting package metadata (repodata.json): ...working... done #0 55.44 Solving environment: ...working... failed #0 65.63 #0 65.63 LibMambaUnsatisfiableError: Encountered problems while solving: #0 65.63 - package libmambapy-1.5.3-py311h2dafd23_0 requires openssl >=3.0.11,<4.0a0, but none of the providers can be installed #0 65.63 #0 65.63 Could not solve for environment specs #0 65.63 The following packages are incompatible #0 65.63 ├─ conda 23.11.0 is installable with the potential options #0 65.63 │ ├─ conda 23.11.0 would require #0 65.63 │ │ └─ python >=3.11,<3.12.0a0 , which can be installed; #0 65.63 │ ├─ conda 23.11.0 would require #0 65.63 │ │ └─ python_abi 3.10.* *_cp310, which can be installed; #0 65.63 │ ├─ conda 23.11.0 would require #0 65.63 │ │ └─ python_abi 3.11.* *_cp311, which can be installed; #0 65.63 │ ├─ conda 23.11.0 would require #0 65.63 │ │ └─ python_abi 3.12.* *_cp312, which can be installed; #0 65.63 │ ├─ conda 23.11.0 would require #0 65.63 │ │ └─ python_abi 3.8.* *_cp38, which can be installed; #0 65.63 │ ├─ conda 23.11.0 would require #0 65.63 │ │ └─ pypy3.9 >=7.3.13 , which can be installed; #0 65.63 │ ├─ conda 23.11.0 would require #0 65.63 │ │ └─ conda-libmamba-solver >=23.11.0 with the potential options #0 65.63 │ │ ├─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 65.63 │ │ │ └─ libmambapy >=1.5.3 with the potential options #0 65.63 │ │ │ ├─ libmambapy 1.5.3 would require #0 65.63 │ │ │ │ └─ openssl >=3.0.11,<4.0a0 , which can be installed; #0 65.63 │ │ │ ├─ libmambapy 1.5.3 would require #0 65.63 │ │ │ │ └─ openssl >=3.1.4,<4.0a0 , which can be installed; #0 65.63 │ │ │ ├─ libmambapy [1.5.4|1.5.5|1.5.6] would require #0 65.63 │ │ │ │ └─ openssl >=3.2.0,<4.0a0 , which can be installed; #0 65.63 │ │ │ └─ libmambapy [1.5.3|1.5.6] would require #0 65.63 │ │ │ └─ openssl >=3.0.12,<4.0a0 , which can be installed; #0 65.63 │ │ ├─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 65.63 │ │ │ └─ python >=3.10,<3.11.0a0 , which can be installed; #0 65.63 │ │ ├─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 65.63 │ │ │ └─ python >=3.11,<3.12.0a0 , which can be installed; #0 65.63 │ │ └─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 65.63 │ │ └─ python >=3.8,<3.9.0a0 , which can be installed; #0 65.63 │ ├─ conda 23.11.0 would require #0 65.63 │ │ └─ python >=3.10,<3.11.0a0 , which can be installed; #0 65.63 │ └─ conda 23.11.0 would require #0 65.63 │ └─ python >=3.8,<3.9.0a0 , which can be installed; #0 65.63 ├─ openssl ==1.1.1w hd590300_0 is not installable because it conflicts with any installable versions previously reported; #0 65.63 └─ python ==3.9.0 hffdb5ce_5_cpython is not installable because it requires #0 65.63 └─ python_abi 3.9.* *_cp39, which conflicts with any installable versions previously reported. #0 65.63 ------ Dockerfile:39 -------------------- 38 | COPY environment_minimal.yaml /root 39 | >>> RUN conda env update -n base -f /root/environment_minimal.yaml && \ 40 | >>> rm /root/environment_minimal.yaml && \ 41 | >>> cd /root && \ 42 | >>> python -m mani_skill2.utils.download_asset all -y && \ 43 | >>> conda clean -ya && \ 44 | >>> pip cache purge 45 | -------------------- ERROR: failed to solve: process "/bin/bash -c conda env update -n base -f /root/environment_minimal.yaml && rm /root/environment_minimal.yaml && cd /root && python -m mani_skill2.utils.download_asset all -y && conda clean -ya && pip cache purge" did not complete successfully: exit code: 1 ```

nicklashansen commented 8 months ago

@josephrocca Got it, thanks! I will see if I can build the docker image on my end and get back to you soon.

nicklashansen commented 8 months ago

Conda also fails to solve on my end when building through docker, but works outside of docker for some reason. It might take a little while for me to find the cause. In the meantime, you can consider using my own (mostly internal) image available here. It supports DMControl, Meta-World, ManiSkill2 as-is, and support for MyoSuite can be added with a pip install myosuite (this breaks the other environments though due to incompatible gym versions).

josephrocca commented 8 months ago

Thanks! This seems to be working, albeit with a slightly older version of the code. Replication, for @ryanwhite04 and any others visiting this thread:

docker run --rm -it --gpus all -v /home/foo/tdmpc2_mnt:/mnt nicklashansen/tdmpcv2:0.1.2 bash
cd /mnt
git clone https://github.com/nicklashansen/tdmpc2
cd tdmpc2
# need to go back to Dec 22nd 2023 due to SliceBuffer errors caused by this: https://github.com/nicklashansen/tdmpc2/commit/3ded0ebc83bbc5480ccba9aab5768688a0c38542
git reset --hard 445af9d81d9f459ebeec4f43995ede2ee573e1fd
# remove mentions of metaworld (causes "No module named 'metaworld'" error):
sed -i 's/make_metaworld_env,//g' /mnt/tdmpc2/tdmpc2/envs/__init__.py
sed -i 's/from envs.metaworld import make_env as make_metaworld_env//g' /mnt/tdmpc2/tdmpc2/envs/__init__.py
# then this works (or at least begins training without errors):
cd tdmpc2
python train.py task=dog-run steps=700000

I will update this thread if I work out how to get the latest version of this repo working (distributed stuff, pixel observations, etc). If you have a newer version of the docker image that supports the latest changes to this repo, that'd be great!

nicklashansen commented 8 months ago

@josephrocca @ryanwhite04 Success! Commit https://github.com/nicklashansen/tdmpc2/commit/e8f1ed6785741eb4c9cabbba50657342520c143b updates the build instructions with an updated dockerfile + conda env that builds for me, both with and without docker. The unmaintained gym dependency is really painful to install at this point, but I have pinned versions now so that should mitigate it a little bit.

josephrocca commented 8 months ago

@nicklashansen Thanks for your efforts in making your research so reproducible! This is a very valuable public good - you've raised the bar here. I've been having a lot of fun training agents and fiddling with the codebase this past week.

RE the new commit - I just tried this:

git clone https://github.com/nicklashansen/tdmpc2
cd tdmpc2
cd docker && docker build . -t josephrocca/tdmpc2:1.0.0

and got a similar LibMambaUnsatisfiableError error:

Click for logs

``` cd docker && docker build . -t josephrocca/tdmpc2:1.0.0 [+] Building 138.8s (10/15) => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 3.01kB 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/nvidia/cudagl:11.3.1-devel-ubuntu20.04 0.0s => [ 1/11] FROM docker.io/nvidia/cudagl:11.3.1-devel-ubuntu20.04 0.0s => [internal] load build context 0.0s => => transferring context: 1.88kB 0.0s => CACHED [ 2/11] RUN apt-get -y update && apt-get install -y --no-install-recommends build-essential gi 0.0s => CACHED [ 3/11] RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O 0.0s => CACHED [ 4/11] COPY nvidia_icd.json /usr/share/vulkan/icd.d/nvidia_icd.json 0.0s => [ 5/11] COPY environment.yaml /root 0.0s => ERROR [ 6/11] RUN conda update conda && conda env update -n base -f /root/environment.yaml && r 138.7s ------ > [ 6/11] RUN conda update conda && conda env update -n base -f /root/environment.yaml && rm /root/environment.yaml && conda clean -ya && pip cache purge: #0 1.354 Channels: #0 1.354 - defaults #0 1.354 Platform: linux-64 #0 1.354 Collecting package metadata (repodata.json): ...working... done #0 9.141 Solving environment: ...working... done #0 9.339 #0 9.339 # All requested packages already installed. #0 9.339 #0 10.44 Channels: #0 10.44 - pytorch-nightly #0 10.44 - nvidia #0 10.44 - conda-forge #0 10.44 - defaults #0 10.44 Platform: linux-64 #0 10.44 Collecting package metadata (repodata.json): ...working... done #0 52.72 Solving environment: ...working... failed #0 90.78 Channels: #0 90.78 - pytorch-nightly #0 90.78 - nvidia #0 90.78 - conda-forge #0 90.78 - defaults #0 90.78 Platform: linux-64 #0 90.78 Collecting package metadata (repodata.json): ...working... done #0 109.5 Solving environment: ...working... failed #0 138.5 #0 138.5 LibMambaUnsatisfiableError: Encountered problems while solving: #0 138.5 - package conda-23.11.0-py311h06a4308_0 requires python >=3.11,<3.12.0a0, but none of the providers can be installed #0 138.5 #0 138.5 Could not solve for environment specs #0 138.5 The following packages are incompatible #0 138.5 ├─ conda 23.11.0 is installable with the potential options #0 138.5 │ ├─ conda 23.11.0 would require #0 138.5 │ │ └─ python >=3.11,<3.12.0a0 , which can be installed; #0 138.5 │ ├─ conda 23.11.0 would require #0 138.5 │ │ ├─ python >=3.10,<3.11.0a0 , which can be installed; #0 138.5 │ │ └─ python_abi 3.10.* *_cp310 with the potential options #0 138.5 │ │ ├─ python_abi 3.10, which can be installed; #0 138.5 │ │ └─ python_abi 3.10 would require #0 138.5 │ │ └─ python 3.10.* *_cpython, which can be installed; #0 138.5 │ ├─ conda 23.11.0 would require #0 138.5 │ │ ├─ python >=3.11,<3.12.0a0 , which can be installed; #0 138.5 │ │ └─ python_abi 3.11.* *_cp311 with the potential options #0 138.5 │ │ ├─ python_abi 3.11, which can be installed; #0 138.5 │ │ └─ python_abi 3.11 would require #0 138.5 │ │ └─ python 3.11.* *_cpython, which can be installed; #0 138.5 │ ├─ conda 23.11.0 would require #0 138.5 │ │ └─ python_abi 3.12.* *_cp312, which requires #0 138.5 │ │ └─ python 3.12.* *_cpython, which can be installed; #0 138.5 │ ├─ conda 23.11.0 would require #0 138.5 │ │ ├─ python >=3.8,<3.9.0a0 , which can be installed; #0 138.5 │ │ └─ python_abi 3.8.* *_cp38 with the potential options #0 138.5 │ │ ├─ python_abi 3.8, which can be installed; #0 138.5 │ │ └─ python_abi 3.8 would require #0 138.5 │ │ └─ python 3.8.* *_cpython, which can be installed; #0 138.5 │ ├─ conda 23.11.0 would require #0 138.5 │ │ └─ pypy3.9 >=7.3.13 , which requires #0 138.5 │ │ └─ python 3.9.* *_73_pypy, which can be installed; #0 138.5 │ ├─ conda 23.11.0 would require #0 138.5 │ │ └─ conda-libmamba-solver >=23.11.0 with the potential options #0 138.5 │ │ ├─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 138.5 │ │ │ └─ libmambapy >=1.5.3 with the potential options #0 138.5 │ │ │ ├─ libmambapy 1.5.3 would require #0 138.5 │ │ │ │ └─ openssl >=3.0.11,<4.0a0 , which can be installed; #0 138.5 │ │ │ ├─ libmambapy [1.5.3|1.5.6] would require #0 138.5 │ │ │ │ └─ python >=3.11,<3.12.0a0 , which can be installed; #0 138.5 │ │ │ ├─ libmambapy [1.5.3|1.5.4|1.5.5|1.5.6] would require #0 138.5 │ │ │ │ ├─ python >=3.10,<3.11.0a0 , which can be installed; #0 138.5 │ │ │ │ └─ python_abi 3.10.* *_cp310 with the potential options #0 138.5 │ │ │ │ ├─ python_abi 3.10, which can be installed; #0 138.5 │ │ │ │ └─ python_abi 3.10, which can be installed (as previously explained); #0 138.5 │ │ │ ├─ libmambapy [1.5.3|1.5.4|1.5.5|1.5.6] would require #0 138.5 │ │ │ │ ├─ python >=3.11,<3.12.0a0 , which can be installed; #0 138.5 │ │ │ │ └─ python_abi 3.11.* *_cp311 with the potential options #0 138.5 │ │ │ │ ├─ python_abi 3.11, which can be installed; #0 138.5 │ │ │ │ └─ python_abi 3.11, which can be installed (as previously explained); #0 138.5 │ │ │ ├─ libmambapy [1.5.3|1.5.4|1.5.5|1.5.6] would require #0 138.5 │ │ │ │ └─ python_abi 3.12.* *_cp312, which can be installed (as previously explained); #0 138.5 │ │ │ ├─ libmambapy [1.5.3|1.5.4|1.5.5|1.5.6] would require #0 138.5 │ │ │ │ ├─ python >=3.8,<3.9.0a0 , which can be installed; #0 138.5 │ │ │ │ └─ python_abi 3.8.* *_cp38 with the potential options #0 138.5 │ │ │ │ ├─ python_abi 3.8, which can be installed; #0 138.5 │ │ │ │ └─ python_abi 3.8, which can be installed (as previously explained); #0 138.5 │ │ │ ├─ libmambapy 1.5.3 would require #0 138.5 │ │ │ │ └─ openssl >=3.1.4,<4.0a0 , which can be installed; #0 138.5 │ │ │ ├─ libmambapy [1.5.3|1.5.4|1.5.5|1.5.6] would require #0 138.5 │ │ │ │ └─ pypy3.9 >=7.3.13 , which can be installed (as previously explained); #0 138.5 │ │ │ ├─ libmambapy [1.5.4|1.5.5|1.5.6] would require #0 138.5 │ │ │ │ └─ openssl >=3.2.0,<4.0a0 , which can be installed; #0 138.5 │ │ │ ├─ libmambapy [1.5.3|1.5.6] would require #0 138.5 │ │ │ │ └─ python >=3.10,<3.11.0a0 , which can be installed; #0 138.5 │ │ │ ├─ libmambapy 1.5.3 would require #0 138.5 │ │ │ │ └─ python >=3.12,<3.13.0a0 , which can be installed; #0 138.5 │ │ │ ├─ libmambapy [1.5.3|1.5.6] would require #0 138.5 │ │ │ │ └─ python >=3.8,<3.9.0a0 , which can be installed; #0 138.5 │ │ │ └─ libmambapy 1.5.6 would require #0 138.5 │ │ │ └─ openssl >=3.0.12,<4.0a0 , which can be installed; #0 138.5 │ │ ├─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 138.5 │ │ │ └─ python >=3.10,<3.11.0a0 , which can be installed; #0 138.5 │ │ ├─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 138.5 │ │ │ └─ python >=3.11,<3.12.0a0 , which can be installed; #0 138.5 │ │ └─ conda-libmamba-solver [23.11.0|23.11.1|23.12.0] would require #0 138.5 │ │ └─ python >=3.8,<3.9.0a0 , which can be installed; #0 138.5 │ ├─ conda 23.11.0 would require #0 138.5 │ │ └─ python >=3.10,<3.11.0a0 , which can be installed; #0 138.5 │ └─ conda 23.11.0 would require #0 138.5 │ └─ python >=3.8,<3.9.0a0 , which can be installed; #0 138.5 └─ python 3.9.0** is not installable because there are no viable options #0 138.5 ├─ python 3.9.0 would require #0 138.5 │ └─ openssl >=1.1.1h,<1.1.2a , which conflicts with any installable versions previously reported; #0 138.5 └─ python 3.9.0 would require #0 138.5 ├─ openssl >=1.1.1h,<1.1.2a , which conflicts with any installable versions previously reported; #0 138.5 └─ python_abi 3.9.* *_cp39, which conflicts with any installable versions previously reported. #0 138.5 ------ Dockerfile:47 -------------------- 46 | COPY environment.yaml /root 47 | >>> RUN conda update conda && \ 48 | >>> conda env update -n base -f /root/environment.yaml && \ 49 | >>> rm /root/environment.yaml && \ 50 | >>> conda clean -ya && \ 51 | >>> pip cache purge 52 | -------------------- ERROR: failed to solve: process "/bin/bash -c conda update conda && conda env update -n base -f /root/environment.yaml && rm /root/environment.yaml && conda clean -ya && pip cache purge" did not complete successfully: exit code: 1 ```

The relevant part of the logs:

#0 138.5 LibMambaUnsatisfiableError: Encountered problems while solving:
#0 138.5   - package conda-23.11.0-py311h06a4308_0 requires python >=3.11,<3.12.0a0, but none of the providers can be installed

Is there anything I'm missing in the commands I ran here? I double-checked that cat environment.yaml matches the new one from your commit, with all the pinned versions.

josephrocca commented 8 months ago

By the way, I had to "flatten" the layers of the prebuilt Docker image that you linked earlier and add this to the new Dockerfile:

RUN find /root/.mujoco -uid 421709 -exec chown root:root {} \;

because the .mujoco directory is owned by a user with uid=421709 instead of root, and Docker cloud services don't like uids above 65536. I think it's something to do with the way they handle virtualization:

So it might be worth adding that command to the Dockerfile at the end of the # mujoco 2.1.0 section so others can easily get it running on cloud machines.

nicklashansen commented 8 months ago

@josephrocca Interesting... I am surprised that there would be a difference between builds when using docker. This is the new image I built: nicklashansen/tdmpc2/1.0.0.

Based on your error message, it seems that the culprit might be the conda or python versions? It should install python=3.9 as specified in the environment file, perhaps downgrading conda might fix your error? I did not have to do anything like that on my end though; I ran the build exactly as is.

Re: ownership of .mujoco I'm seeing root as the owner in my (new) image as well, output of ls -lha:

drwxr-xr-x  3 root root 4.0K Jan 22 05:07 .mujoco

josephrocca commented 8 months ago

Yeah I'm really surprised too if we're both running the same commands! My understanding was that a Docker build process/behavior should be ~completely independent of the host machine.

RE non-root uids, it's actually /root/.mujoco/mujoco210, rather than /root/.mujoco (my mistake):

$ docker run --rm -it nicklashansen/tdmpcv2:0.1.2 bash
(base) root@b6ebaf5f0141:/# find /root/.mujoco -uid 421709
/root/.mujoco/mujoco210
/root/.mujoco/mujoco210/sample
/root/.mujoco/mujoco210/sample/simulate.cc
/root/.mujoco/mujoco210/sample/testspeed.cc
/root/.mujoco/mujoco210/sample/derivative.cc
/root/.mujoco/mujoco210/sample/record.cc
/root/.mujoco/mujoco210/sample/basic.cc
/root/.mujoco/mujoco210/sample/testxml.cc
/root/.mujoco/mujoco210/sample/compile.cc
/root/.mujoco/mujoco210/sample/Makefile
/root/.mujoco/mujoco210/model
/root/.mujoco/mujoco210/model/softbox.xml
/root/.mujoco/mujoco210/model/grid2.xml
/root/.mujoco/mujoco210/model/rope.xml
/root/.mujoco/mujoco210/model/grid1.xml
/root/.mujoco/mujoco210/model/grid2pin.xml
/root/.mujoco/mujoco210/model/humanoid100.xml
/root/.mujoco/mujoco210/model/softellipsoid.xml
/root/.mujoco/mujoco210/model/humanoid.xml
/root/.mujoco/mujoco210/model/grid1pin.xml
/root/.mujoco/mujoco210/model/sponge.png
/root/.mujoco/mujoco210/model/hammock.xml
/root/.mujoco/mujoco210/model/cloth.xml
/root/.mujoco/mujoco210/model/softcylinder.xml
/root/.mujoco/mujoco210/model/arm26.xml
/root/.mujoco/mujoco210/model/carpet.png
/root/.mujoco/mujoco210/model/particle.xml
/root/.mujoco/mujoco210/model/marble.png
/root/.mujoco/mujoco210/model/loop.xml
/root/.mujoco/mujoco210/model/scene.xml
/root/.mujoco/mujoco210/include
/root/.mujoco/mujoco210/include/mjrender.h
/root/.mujoco/mujoco210/include/mujoco.h
/root/.mujoco/mujoco210/include/mjui.h
/root/.mujoco/mujoco210/include/uitools.h
/root/.mujoco/mujoco210/include/mjvisualize.h
/root/.mujoco/mujoco210/include/glfw3.h
/root/.mujoco/mujoco210/include/mjmodel.h
/root/.mujoco/mujoco210/include/uitools.c
/root/.mujoco/mujoco210/include/mjdata.h
/root/.mujoco/mujoco210/include/mjxmacro.h
/root/.mujoco/mujoco210/THIRD_PARTY_NOTICES
/root/.mujoco/mujoco210/bin
/root/.mujoco/mujoco210/bin/libglew.so
/root/.mujoco/mujoco210/bin/basic
/root/.mujoco/mujoco210/bin/libglfw3.a
/root/.mujoco/mujoco210/bin/libmujoco210.so
/root/.mujoco/mujoco210/bin/libmujoco210nogl.so
/root/.mujoco/mujoco210/bin/libglewosmesa.so
/root/.mujoco/mujoco210/bin/libglewegl.so
/root/.mujoco/mujoco210/bin/testspeed
/root/.mujoco/mujoco210/bin/testxml
/root/.mujoco/mujoco210/bin/simulate
/root/.mujoco/mujoco210/bin/record
/root/.mujoco/mujoco210/bin/compile
/root/.mujoco/mujoco210/bin/libglfw.so.3
/root/.mujoco/mujoco210/bin/derivative

(base) root@b6ebaf5f0141:/# ls -lah /root/.mujoco
total 16K
drwxr-xr-x 3 root   root  4.0K Mar 24  2023 .
drwx------ 1 root   root  4.0K Mar 24  2023 ..
-rw-r--r-- 1 root   root   768 Oct 18  2021 mjkey.txt
drwxr-xr-x 6 421709 89939 4.0K Oct  7  2021 mujoco210
(base) root@b6ebaf5f0141:/# ls -lah /root/.mujoco/mujoco210
total 36K
drwxr-xr-x 6 421709 89939 4.0K Oct  7  2021 .
drwxr-xr-x 3 root   root  4.0K Mar 24  2023 ..
-rw-r--r-- 1 421709 89939  10K Oct  7  2021 THIRD_PARTY_NOTICES
drwxr-xr-x 2 421709 89939 4.0K Oct  3  2021 bin
drwxr-xr-x 2 421709 89939 4.0K Oct  3  2021 include
drwxr-xr-x 2 421709 89939 4.0K Oct  3  2021 model
drwxr-xr-x 2 421709 89939 4.0K Oct 15  2021 sample

I just checked and this is indeed still the case in nicklashansen/tdmpc2:1.0.0. If you could push a 1.0.1 that'd be really handy! Flattening the filesystem layers seems to make the docker image a significantly larger download, since IIUC cloud services can't use the cached nvidia/cudagl:11.3.1-devel-ubuntu20.04 base image (I could be wrong about this, half my Docker knowledge is from ChatGPT).

nicklashansen commented 8 months ago

@josephrocca I pushed a new version of nicklashansen/tdmpc2:1.0.0 which has this change:

(base) root@68ea27fb0e06:~/.mujoco# ls -lha
total 16K
drwxr-xr-x 3 root root 4.0K Jan 22 23:42 .
drwx------ 1 root root 4.0K Jan 22 23:44 ..
-rw-r--r-- 1 root root  750 Jan 21 23:18 mjkey.txt
drwxr-xr-x 6 root root 4.0K Oct  7  2021 mujoco210

You will need to pull the latest version from dockerhub. Let me know if that solves your issue!

josephrocca commented 8 months ago

That solved it - thank you Nicklas!

nicklashansen commented 8 months ago

Great to hear! Let me know if you run into any other issues :-)

nicklashansen / tdmpc2