search

pytorch-labs / LeanRL

LeanRL is a fork of CleanRL, where selected PyTorch scripts optimized for performance using compile and cudagraphs.
Other
432 stars 17 forks source link

Some installation issues #2

Open vwxyzjn opened 1 month ago

vwxyzjn commented 1 month ago

There seem to be some issues with getting the environment set up. I tried two installation methods.

Installation 1

One is to do 

pip install -r requirements/requirements-envpool.txt

which seems to get stuck finding a torchrl_nightly version

Ignoring importlib-metadata: markers 'python_version >= "3.8" and python_version < "3.10"' don't match your environment
Collecting absl-py==1.4.0 (from -r requirements/requirements-envpool.txt (line 1))
  Using cached absl_py-1.4.0-py3-none-any.whl.metadata (2.3 kB)
Collecting appdirs==1.4.4 (from -r requirements/requirements-envpool.txt (line 2))
  Using cached appdirs-1.4.4-py2.py3-none-any.whl.metadata (9.0 kB)
Collecting bitmath==1.3.3.1 (from -r requirements/requirements-envpool.txt (line 3))
  Using cached bitmath-1.3.3.1.tar.gz (88 kB)
  Preparing metadata (setup.py) ... done
Collecting cachetools==5.3.0 (from -r requirements/requirements-envpool.txt (line 4))
  Using cached cachetools-5.3.0-py3-none-any.whl.metadata (5.2 kB)
Collecting chardet==4.0.0 (from -r requirements/requirements-envpool.txt (line 5))
  Using cached chardet-4.0.0-py2.py3-none-any.whl.metadata (3.5 kB)
Collecting charset-normalizer==3.1.0 (from -r requirements/requirements-envpool.txt (line 6))
  Using cached charset_normalizer-3.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (30 kB)
Collecting click==8.1.3 (from -r requirements/requirements-envpool.txt (line 7))
  Using cached click-8.1.3-py3-none-any.whl.metadata (3.2 kB)
Collecting cloudpickle==2.2.1 (from -r requirements/requirements-envpool.txt (line 8))
  Using cached cloudpickle-2.2.1-py3-none-any.whl.metadata (6.9 kB)
Collecting colorama==0.4.4 (from -r requirements/requirements-envpool.txt (line 9))
  Using cached colorama-0.4.4-py2.py3-none-any.whl.metadata (14 kB)
Collecting commonmark==0.9.1 (from -r requirements/requirements-envpool.txt (line 10))
  Using cached commonmark-0.9.1-py2.py3-none-any.whl.metadata (5.7 kB)
Collecting cycler==0.11.0 (from -r requirements/requirements-envpool.txt (line 11))
  Using cached cycler-0.11.0-py3-none-any.whl.metadata (785 bytes)
Collecting decorator==4.4.2 (from -r requirements/requirements-envpool.txt (line 12))
  Using cached decorator-4.4.2-py2.py3-none-any.whl.metadata (4.2 kB)
Collecting dill==0.3.6 (from -r requirements/requirements-envpool.txt (line 13))
  Using cached dill-0.3.6-py3-none-any.whl.metadata (9.8 kB)
Collecting dm-env==1.6 (from -r requirements/requirements-envpool.txt (line 14))
  Using cached dm_env-1.6-py3-none-any.whl.metadata (966 bytes)
Collecting dm-tree==0.1.8 (from -r requirements/requirements-envpool.txt (line 15))
  Using cached dm_tree-0.1.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.9 kB)
Collecting docker-pycreds==0.4.0 (from -r requirements/requirements-envpool.txt (line 16))
  Using cached docker_pycreds-0.4.0-py2.py3-none-any.whl.metadata (1.8 kB)
Collecting docstring-parser==0.15 (from -r requirements/requirements-envpool.txt (line 17))
  Using cached docstring_parser-0.15-py3-none-any.whl.metadata (2.4 kB)
Collecting enum-tools==0.9.0.post1 (from -r requirements/requirements-envpool.txt (line 18))
  Using cached enum_tools-0.9.0.post1-py3-none-any.whl.metadata (7.9 kB)
Collecting envpool==0.6.6 (from -r requirements/requirements-envpool.txt (line 19))
  Using cached envpool-0.6.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (14 kB)
Collecting farama-notifications==0.0.4 (from -r requirements/requirements-envpool.txt (line 20))
  Using cached Farama_Notifications-0.0.4-py3-none-any.whl.metadata (558 bytes)
Collecting filelock==3.12.0 (from -r requirements/requirements-envpool.txt (line 21))
  Using cached filelock-3.12.0-py3-none-any.whl.metadata (2.5 kB)
Collecting gitdb==4.0.10 (from -r requirements/requirements-envpool.txt (line 22))
  Using cached gitdb-4.0.10-py3-none-any.whl.metadata (1.1 kB)
Collecting google-auth-oauthlib==0.4.6 (from -r requirements/requirements-envpool.txt (line 23))
  Using cached google_auth_oauthlib-0.4.6-py2.py3-none-any.whl.metadata (2.7 kB)
Collecting google-auth==2.18.0 (from -r requirements/requirements-envpool.txt (line 24))
  Using cached google_auth-2.18.0-py2.py3-none-any.whl.metadata (4.4 kB)
Collecting graphviz==0.20.1 (from -r requirements/requirements-envpool.txt (line 25))
  Using cached graphviz-0.20.1-py3-none-any.whl.metadata (12 kB)
Collecting gym-notices==0.0.8 (from -r requirements/requirements-envpool.txt (line 26))
  Using cached gym_notices-0.0.8-py3-none-any.whl.metadata (1.0 kB)
Collecting gym==0.23.1 (from -r requirements/requirements-envpool.txt (line 27))
  Using cached gym-0.23.1.tar.gz (626 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting gymnasium==0.28.1 (from -r requirements/requirements-envpool.txt (line 28))
  Using cached gymnasium-0.28.1-py3-none-any.whl.metadata (9.2 kB)
Collecting hbutils==0.8.6 (from -r requirements/requirements-envpool.txt (line 29))
  Using cached hbutils-0.8.6-py3-none-any.whl.metadata (8.2 kB)
Collecting huggingface-hub==0.11.1 (from -r requirements/requirements-envpool.txt (line 30))
  Using cached huggingface_hub-0.11.1-py3-none-any.whl.metadata (7.5 kB)
Collecting imageio-ffmpeg==0.3.0 (from -r requirements/requirements-envpool.txt (line 31))
  Using cached imageio_ffmpeg-0.3.0-py3-none-manylinux2010_x86_64.whl.metadata (1.4 kB)
Collecting imageio==2.28.1 (from -r requirements/requirements-envpool.txt (line 32))
  Using cached imageio-2.28.1-py3-none-any.whl.metadata (4.7 kB)
Collecting jax-jumpy==1.0.0 (from -r requirements/requirements-envpool.txt (line 34))
  Using cached jax_jumpy-1.0.0-py3-none-any.whl.metadata (15 kB)
Collecting kiwisolver==1.4.4 (from -r requirements/requirements-envpool.txt (line 35))
  Using cached kiwisolver-1.4.4-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (6.3 kB)
Collecting markdown==3.3.7 (from -r requirements/requirements-envpool.txt (line 36))
  Using cached Markdown-3.3.7-py3-none-any.whl.metadata (4.6 kB)
Collecting markupsafe==2.1.2 (from -r requirements/requirements-envpool.txt (line 37))
  Using cached MarkupSafe-2.1.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.2 kB)
Collecting matplotlib==3.5.3 (from -r requirements/requirements-envpool.txt (line 38))
  Using cached matplotlib-3.5.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)
Collecting moviepy==1.0.3 (from -r requirements/requirements-envpool.txt (line 39))
  Using cached moviepy-1.0.3.tar.gz (388 kB)
  Preparing metadata (setup.py) ... done
Collecting numpy==1.24.4 (from -r requirements/requirements-envpool.txt (line 40))
  Using cached numpy-1.24.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.6 kB)
Collecting oauthlib==3.2.2 (from -r requirements/requirements-envpool.txt (line 41))
  Using cached oauthlib-3.2.2-py3-none-any.whl.metadata (7.5 kB)
Collecting packaging==23.1 (from -r requirements/requirements-envpool.txt (line 42))
  Using cached packaging-23.1-py3-none-any.whl.metadata (3.1 kB)
Collecting pandas==1.3.5 (from -r requirements/requirements-envpool.txt (line 43))
  Using cached pandas-1.3.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)
Collecting pathtools==0.1.2 (from -r requirements/requirements-envpool.txt (line 44))
  Using cached pathtools-0.1.2.tar.gz (11 kB)
  Preparing metadata (setup.py) ... done
Collecting proglog==0.1.10 (from -r requirements/requirements-envpool.txt (line 45))
  Using cached proglog-0.1.10-py3-none-any.whl.metadata (639 bytes)
Collecting protobuf==3.20.3 (from -r requirements/requirements-envpool.txt (line 46))
  Using cached protobuf-3.20.3-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (679 bytes)
Collecting psutil==5.9.5 (from -r requirements/requirements-envpool.txt (line 47))
  Using cached psutil-5.9.5-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (21 kB)
Collecting pyasn1-modules==0.3.0 (from -r requirements/requirements-envpool.txt (line 48))
  Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl.metadata (3.6 kB)
Collecting pyasn1==0.5.0 (from -r requirements/requirements-envpool.txt (line 49))
  Using cached pyasn1-0.5.0-py2.py3-none-any.whl.metadata (8.5 kB)
Collecting pygame==2.1.0 (from -r requirements/requirements-envpool.txt (line 50))
  Using cached pygame-2.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.5 kB)
Collecting pygments==2.15.1 (from -r requirements/requirements-envpool.txt (line 51))
  Using cached Pygments-2.15.1-py3-none-any.whl.metadata (2.5 kB)
Collecting pyparsing==3.0.9 (from -r requirements/requirements-envpool.txt (line 52))
  Using cached pyparsing-3.0.9-py3-none-any.whl.metadata (4.2 kB)
Collecting python-dateutil==2.8.2 (from -r requirements/requirements-envpool.txt (line 53))
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl.metadata (8.2 kB)
Collecting pytimeparse==1.1.8 (from -r requirements/requirements-envpool.txt (line 54))
  Using cached pytimeparse-1.1.8-py2.py3-none-any.whl.metadata (3.4 kB)
Collecting pytz==2023.3 (from -r requirements/requirements-envpool.txt (line 55))
  Using cached pytz-2023.3-py2.py3-none-any.whl.metadata (22 kB)
Collecting pyyaml==6.0.1 (from -r requirements/requirements-envpool.txt (line 56))
  Using cached PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Collecting rich==11.2.0 (from -r requirements/requirements-envpool.txt (line 57))
  Using cached rich-11.2.0-py3-none-any.whl.metadata (19 kB)
Collecting rsa==4.7.2 (from -r requirements/requirements-envpool.txt (line 58))
  Using cached rsa-4.7.2-py3-none-any.whl.metadata (3.6 kB)
Collecting setproctitle==1.3.2 (from -r requirements/requirements-envpool.txt (line 59))
  Using cached setproctitle-1.3.2-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.8 kB)
Collecting shtab==1.6.4 (from -r requirements/requirements-envpool.txt (line 60))
  Using cached shtab-1.6.4-py3-none-any.whl.metadata (7.3 kB)
Collecting six==1.16.0 (from -r requirements/requirements-envpool.txt (line 61))
  Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Collecting smmap==5.0.0 (from -r requirements/requirements-envpool.txt (line 62))
  Using cached smmap-5.0.0-py3-none-any.whl.metadata (4.2 kB)
Collecting stable-baselines3==2.0.0 (from -r requirements/requirements-envpool.txt (line 63))
  Using cached stable_baselines3-2.0.0-py3-none-any.whl.metadata (5.4 kB)
Collecting tenacity==8.2.3 (from -r requirements/requirements-envpool.txt (line 64))
  Using cached tenacity-8.2.3-py3-none-any.whl.metadata (1.0 kB)
Collecting tensorboard-data-server==0.6.1 (from -r requirements/requirements-envpool.txt (line 65))
  Using cached tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl.metadata (1.1 kB)
Collecting tensorboard-plugin-wit==1.8.1 (from -r requirements/requirements-envpool.txt (line 66))
  Using cached tensorboard_plugin_wit-1.8.1-py3-none-any.whl.metadata (873 bytes)
Collecting tensorboard==2.11.2 (from -r requirements/requirements-envpool.txt (line 67))
  Using cached tensorboard-2.11.2-py3-none-any.whl.metadata (1.9 kB)
Collecting tqdm (from -r requirements/requirements-envpool.txt (line 68))
  Using cached tqdm-4.66.5-py3-none-any.whl.metadata (57 kB)
Collecting treevalue==1.4.10 (from -r requirements/requirements-envpool.txt (line 69))
  Using cached treevalue-1.4.10-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (22 kB)
Collecting types-protobuf==4.23.0.1 (from -r requirements/requirements-envpool.txt (line 70))
  Using cached types_protobuf-4.23.0.1-py3-none-any.whl.metadata (1.3 kB)
Collecting typing-extensions==4.5.0 (from -r requirements/requirements-envpool.txt (line 71))
  Using cached typing_extensions-4.5.0-py3-none-any.whl.metadata (8.5 kB)
Collecting tyro==0.5.10 (from -r requirements/requirements-envpool.txt (line 72))
  Using cached tyro-0.5.10-py3-none-any.whl.metadata (7.1 kB)
Collecting wandb==0.13.11 (from -r requirements/requirements-envpool.txt (line 73))
  Using cached wandb-0.13.11-py3-none-any.whl.metadata (7.9 kB)
Collecting wheel==0.40.0 (from -r requirements/requirements-envpool.txt (line 74))
  Using cached wheel-0.40.0-py3-none-any.whl.metadata (2.1 kB)
Collecting torchrl-nightly (from -r requirements/requirements-envpool.txt (line 75))
  Using cached torchrl_nightly-2024.9.19-cp310-cp310-manylinux1_x86_64.whl.metadata (39 kB)
Collecting tensordict-nightly (from -r requirements/requirements-envpool.txt (line 76))
  Using cached tensordict_nightly-2024.9.19-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
Collecting requests-oauthlib>=0.7.0 (from google-auth-oauthlib==0.4.6->-r requirements/requirements-envpool.txt (line 23))
  Using cached requests_oauthlib-2.0.0-py2.py3-none-any.whl.metadata (11 kB)
Collecting urllib3<2.0 (from google-auth==2.18.0->-r requirements/requirements-envpool.txt (line 24))
  Using cached urllib3-1.26.20-py2.py3-none-any.whl.metadata (50 kB)
Requirement already satisfied: setuptools>=50.0 in /home/costa/.pyenv/versions/miniforge3-22.11.1-4/envs/leanrl3/lib/python3.10/site-packages (from hbutils==0.8.6->-r requirements/requirements-envpool.txt (line 29)) (74.1.2)
Collecting requests (from huggingface-hub==0.11.1->-r requirements/requirements-envpool.txt (line 30))
  Using cached requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting pillow>=8.3.2 (from imageio==2.28.1->-r requirements/requirements-envpool.txt (line 32))
  Using cached pillow-10.4.0-cp310-cp310-manylinux_2_28_x86_64.whl.metadata (9.2 kB)
Collecting fonttools>=4.22.0 (from matplotlib==3.5.3->-r requirements/requirements-envpool.txt (line 38))
  Using cached fonttools-4.53.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (162 kB)
Collecting torch>=1.11 (from stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached torch-2.4.1-cp310-cp310-manylinux1_x86_64.whl.metadata (26 kB)
Collecting grpcio>=1.24.3 (from tensorboard==2.11.2->-r requirements/requirements-envpool.txt (line 67))
  Using cached grpcio-1.66.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.9 kB)
Collecting werkzeug>=1.0.1 (from tensorboard==2.11.2->-r requirements/requirements-envpool.txt (line 67))
  Using cached werkzeug-3.0.4-py3-none-any.whl.metadata (3.7 kB)
Collecting GitPython!=3.1.29,>=1.0.0 (from wandb==0.13.11->-r requirements/requirements-envpool.txt (line 73))
  Using cached GitPython-3.1.43-py3-none-any.whl.metadata (13 kB)
Collecting sentry-sdk>=1.0.0 (from wandb==0.13.11->-r requirements/requirements-envpool.txt (line 73))
  Using cached sentry_sdk-2.14.0-py2.py3-none-any.whl.metadata (9.7 kB)
INFO: pip is looking at multiple versions of torchrl-nightly to determine which version is compatible with other requirements. This could take a while.
Collecting torchrl-nightly (from -r requirements/requirements-envpool.txt (line 75))
  Using cached torchrl_nightly-2024.9.16-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.9.15-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.9.14-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.9.13-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.9.12-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.9.11-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.9.10-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
INFO: pip is still looking at multiple versions of torchrl-nightly to determine which version is compatible with other requirements. This could take a while.
  Using cached torchrl_nightly-2024.9.9-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.9.8-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.9.7-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.9.6-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.9.5-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
  Using cached torchrl_nightly-2024.9.4-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.9.3-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.9.2-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.9.1-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.31-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.30-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.29-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.28-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.27-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.26-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.25-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.24-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.23-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.22-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.21-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.20-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.19-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.18-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.17-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.16-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.15-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.14-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.13-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.12-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.11-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.10-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.9-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.8-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.7-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.6-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.5-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.4-cp310-cp310-manylinux1_x86_64.whl.metadata (34 kB)
  Using cached torchrl_nightly-2024.8.3-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.8.2-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
INFO: pip is looking at multiple versions of tensordict-nightly to determine which version is compatible with other requirements. This could take a while.
Collecting tensordict-nightly (from -r requirements/requirements-envpool.txt (line 76))
  Using cached tensordict_nightly-2024.9.18-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.9.16-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.9.15-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.9.14-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.9.13-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.9.12-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.9.11-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
INFO: pip is still looking at multiple versions of tensordict-nightly to determine which version is compatible with other requirements. This could take a while.
  Downloading tensordict_nightly-2024.9.10-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.9.9-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.9.8-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.9.7-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.9.6-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
  Downloading tensordict_nightly-2024.9.5-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.9.4-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.9.3-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.9.2-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.9.1-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.31-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.30-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.29-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.28-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.27-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.26-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.25-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.24-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.23-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.22-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.21-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.20-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.19-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.18-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.17-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.16-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.15-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.14-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.13-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.12-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.11-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.10-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.9-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.8-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.7-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.6-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.5-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.4-cp310-cp310-manylinux1_x86_64.whl.metadata (9.0 kB)
  Downloading tensordict_nightly-2024.8.3-cp310-cp310-manylinux1_x86_64.whl.metadata (22 kB)
  Downloading tensordict_nightly-2024.8.2-cp310-cp310-manylinux1_x86_64.whl.metadata (22 kB)
  Downloading tensordict_nightly-2024.8.1-cp310-cp310-manylinux1_x86_64.whl.metadata (22 kB)
  Downloading tensordict_nightly-2024.7.3-cp310-cp310-manylinux1_x86_64.whl.metadata (22 kB)
Collecting orjson (from tensordict-nightly->-r requirements/requirements-envpool.txt (line 76))
  Using cached orjson-3.10.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (50 kB)
Collecting idna<4,>=2.5 (from requests->huggingface-hub==0.11.1->-r requirements/requirements-envpool.txt (line 30))
  Using cached idna-3.10-py3-none-any.whl.metadata (10 kB)
Collecting certifi>=2017.4.17 (from requests->huggingface-hub==0.11.1->-r requirements/requirements-envpool.txt (line 30))
  Using cached certifi-2024.8.30-py3-none-any.whl.metadata (2.2 kB)
INFO: pip is looking at multiple versions of torch to determine which version is compatible with other requirements. This could take a while.
Collecting torch>=1.11 (from stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached torch-2.4.0-cp310-cp310-manylinux1_x86_64.whl.metadata (26 kB)
  Using cached torch-2.3.1-cp310-cp310-manylinux1_x86_64.whl.metadata (26 kB)
  Using cached torch-2.3.0-cp310-cp310-manylinux1_x86_64.whl.metadata (26 kB)
INFO: pip is still looking at multiple versions of torch to determine which version is compatible with other requirements. This could take a while.
  Using cached torch-2.2.2-cp310-cp310-manylinux1_x86_64.whl.metadata (26 kB)
  Using cached torch-2.2.1-cp310-cp310-manylinux1_x86_64.whl.metadata (26 kB)
  Using cached torch-2.2.0-cp310-cp310-manylinux1_x86_64.whl.metadata (25 kB)
  Using cached torch-2.1.2-cp310-cp310-manylinux1_x86_64.whl.metadata (25 kB)
Collecting sympy (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Downloading sympy-1.13.3-py3-none-any.whl.metadata (12 kB)
Collecting networkx (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached networkx-3.3-py3-none-any.whl.metadata (5.1 kB)
Collecting jinja2 (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB)
Collecting fsspec (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Downloading fsspec-2024.9.0-py3-none-any.whl.metadata (11 kB)
Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cublas-cu12==12.1.3.1 (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cufft-cu12==11.0.2.54 (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-curand-cu12==10.3.2.106 (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-nccl-cu12==2.18.1 (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached nvidia_nccl_cu12-2.18.1-py3-none-manylinux1_x86_64.whl.metadata (1.8 kB)
Collecting nvidia-nvtx-cu12==12.1.105 (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.7 kB)
Collecting triton==2.1.0 (from torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Using cached triton-2.1.0-0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.3 kB)
Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.11->stable-baselines3==2.0.0->-r requirements/requirements-envpool.txt (line 63))
  Downloading nvidia_nvjitlink_cu12-12.6.68-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting torchrl-nightly (from -r requirements/requirements-envpool.txt (line 75))
  Using cached torchrl_nightly-2024.8.1-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
  Using cached torchrl_nightly-2024.7.31-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.30-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.29-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.28-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.27-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.26-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.25-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.24-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.23-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.22-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.21-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.20-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.19-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.18-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.17-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.16-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.15-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.14-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.13-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.12-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.11-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.10-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.9-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.7.3-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.23-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.22-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.21-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.20-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.19-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.18-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.17-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.16-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.15-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.14-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.13-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.12-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.11-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.10-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.9-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.3-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.2-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.6.1-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.5.31-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.5.30-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.5.29-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.5.28-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.5.27-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.5.26-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.5.25-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.5.24-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.5.23-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.5.22-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.5.21-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.5.20-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.5.19-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Using cached torchrl_nightly-2024.5.18-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Downloading torchrl_nightly-2024.5.17-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Downloading torchrl_nightly-2024.5.16-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Downloading torchrl_nightly-2024.5.15-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Downloading torchrl_nightly-2024.5.14-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Downloading torchrl_nightly-2024.5.13-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Downloading torchrl_nightly-2024.5.12-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Downloading torchrl_nightly-2024.5.11-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Downloading torchrl_nightly-2024.5.10-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Downloading torchrl_nightly-2024.5.9-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Downloading torchrl_nightly-2024.5.8-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Downloading torchrl_nightly-2024.5.7-cp310-cp310-manylinux1_x86_64.whl.metadata (33 kB)
  Downloading torchrl_nightly-2024.5.6-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.5.5-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.5.4-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.5.3-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.5.2-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.5.1-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.4.30-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.4.29-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.4.28-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.4.27-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.4.26-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.4.25-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.4.24-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.4.2-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.4.1-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.3.31-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.3.29-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.3.27-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.3.26-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.3.25-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.3.24-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.3.23-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.3.22-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)
  Downloading torchrl_nightly-2024.3.21-cp310-cp310-manylinux1_x86_64.whl.metadata (31 kB)

Installation 2

I tried building an environment from scratch, with two separate machines with different GPUs, all getting the same error as follows:

absl-py==2.1.0
certifi==2024.8.30
charset-normalizer==3.3.2
click==8.1.7
cloudpickle==3.0.0
dm-env==1.6
dm-tree==0.1.8
docker-pycreds==0.4.0
docstring_parser==0.16
envpool==0.8.4
Farama-Notifications==0.0.4
filelock==3.13.1
fsspec==2024.6.1
gitdb==4.0.11
GitPython==3.1.43
gym==0.23.1
gym-notices==0.0.8
gymnasium==0.28.1
idna==3.10
jax-jumpy==1.0.0
Jinja2==3.1.4
markdown-it-py==3.0.0
MarkupSafe==2.1.5
mdurl==0.1.2
mpmath==1.3.0
networkx==3.3
numpy==2.0.2
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==9.1.0.70
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-nccl-cu12==2.21.5
nvidia-nvjitlink-cu12==12.1.105
nvidia-nvtx-cu12==12.1.105
optree==0.12.1
orjson==3.10.7
packaging==24.1
platformdirs==4.3.6
protobuf==5.28.2
psutil==6.0.0
Pygments==2.18.0
pytorch-triton==3.1.0+5fe38ffd73
PyYAML==6.0.2
requests==2.32.3
rich==13.8.1
sentry-sdk==2.14.0
setproctitle==1.3.3
shtab==1.7.1
six==1.16.0
smmap==5.0.1
sympy==1.13.1
tensordict-nightly==2024.9.19
torch==2.5.0.dev20240911+cu121
torchrl-nightly==2024.9.19
tqdm==4.66.5
types-protobuf==5.27.0.20240907
typing_extensions==4.12.2
tyro==0.8.11
urllib3==2.2.3
wandb==0.18.1
Traceback (most recent call last):
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/leanrl/ppo_atari_envpool_torchcompile.py", line 378, in <module>
    next_obs, next_done, container = rollout(next_obs, next_done, avg_returns=avg_returns)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/leanrl/ppo_atari_envpool_torchcompile.py", line 207, in rollout
    next_obs, reward, next_done, action, logprob, value = act_and_step_func(obs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 465, in _fn
    return fn(*args, **kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/leanrl/ppo_atari_envpool_torchcompile.py", line 193, in act_and_step_func
    action, logprob, _, value = policy(obs=obs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1292, in __call__
    return self._torchdynamo_orig_callable(
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 1087, in __call__
    result = self._inner_convert(
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 530, in __call__
    return _compile(
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 933, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 675, in compile_inner
    return _compile_inner(code, one_graph, hooks, transform)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_utils_internal.py", line 87, in wrapper_function
    return function(*args, **kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 708, in _compile_inner
    out_code = transform_code_object(code, transform)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object
    transformations(instructions, code_options)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 220, in _fn
    return fn(*args, **kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 643, in transform
    tracer.run()
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2776, in run
    super().run()
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run
    while self.step():
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 569, in wrapper
    return inner_fn(self, inst)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1598, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function
    self.push(fn.call_function(self, args, kwargs))  # type: ignore[arg-type]
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/variables/user_defined.py", line 938, in call_function
    return self.call_method(tx, "__call__", args, kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/variables/user_defined.py", line 798, in call_method
    return UserMethodVariable(method, self, source=source).call_function(
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 400, in call_function
    return super().call_function(tx, args, kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 339, in call_function
    return super().call_function(tx, args, kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 111, in call_function
    return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2991, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 3119, in inline_call_
    tracer.run()
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run
    while self.step():
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step
    self.dispatch_table[inst.opcode](self, inst)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 569, in wrapper
    return inner_fn(self, inst)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1676, in CALL_FUNCTION_EX
    self.call_function(fn, argsvars.items, kwargsvars)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function
    self.push(fn.call_function(self, args, kwargs))  # type: ignore[arg-type]
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/variables/lazy.py", line 156, in realize_and_forward
    return getattr(self.realize(), name)(*args, **kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/variables/torch.py", line 953, in call_function
    tensor_variable = wrap_fx_proxy(
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 2045, in wrap_fx_proxy
    return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 2132, in wrap_fx_proxy_cls
    example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 2103, in get_fake_value
    raise TorchRuntimeError(str(e)).with_traceback(e.__traceback__) from None
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 2038, in get_fake_value
    ret_val = wrap_fake_exception(
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1595, in wrap_fake_exception
    return fn()
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 2039, in <lambda>
    lambda: run_node(tx.output, node, args, kwargs, nnmodule)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 2171, in run_node
    raise RuntimeError(make_error_message(e)).with_traceback(
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 2153, in run_node
    return node.target(*args, **kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_ops.py", line 716, in __call__
    return self._op(*args, **kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_library/autograd.py", line 113, in autograd_impl
    result = forward_no_grad(*args, Metadata(keyset, keyword_only_args))
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_library/autograd.py", line 40, in forward_no_grad
    result = op.redispatch(keyset & _C._after_autograd_keyset, *args, **kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_ops.py", line 721, in redispatch
    return self._handle.redispatch_boxed(keyset, *args, **kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/utils/_stats.py", line 21, in wrapper
    return fn(*args, **kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1238, in __torch_dispatch__
    return self.dispatch(func, types, args, kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1692, in dispatch
    return self._cached_dispatch_impl(func, types, args, kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1348, in _cached_dispatch_impl
    output = self._dispatch_impl(func, types, args, kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 2001, in _dispatch_impl
    result = maybe_fake_impl(*args, **kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_library/utils.py", line 20, in __call__
    return self.func(*args, **kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/library.py", line 1156, in inner
    return func(*args, **kwargs)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_library/custom_ops.py", line 608, in fake_impl
    raise RuntimeError(
torch._dynamo.exc.TorchRuntimeError: Failed running call_function mylib.step.default(*(FakeTensor(..., device='cuda:0', size=(8,), dtype=torch.int64),), **{}):
There was no fake impl registered for <CustomOpDef(mylib::step)>. This is necessary for torch.compile/export/fx tracing to work. Please use `step_func.register_fake` to add an fake impl.

from user code:
   File "/net/nfs.cirrascale/allennlp/costa/LeanRL/leanrl/ppo_atari_envpool_torchcompile.py", line 196, in torch_dynamo_resume_in_act_and_step_func_at_193
    next_obs, reward, next_done, info = step_func(action)
  File "/net/nfs.cirrascale/allennlp/costa/LeanRL/.venv/lib/python3.10/site-packages/torch/_library/custom_ops.py", line 669, in __call__
    return self._opoverload(*args, **kwargs)

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information

You can suppress this exception and fall back to eager by setting:
    import torch._dynamo
    torch._dynamo.config.suppress_errors = True
vwxyzjn commented 1 month ago

To fix the default installation, I tried removing # torchrl-nightly # tensordict-nightly from requirements/requirements-envpool.txt and then install from it.

Then I was able to run 

pip install --upgrade --pre torch --index-url https://download.pytorch.org/whl/nightly/cu124
pip install tensordict-nightly
python leanrl/ppo_atari_envpool_torchcompile.py \
    --seed 1 \
    --total-timesteps 50000 \
    --compile \
    --cudagraphs

However still ran into that torch._dynamo.exc.TorchRuntimeError: Failed running call_function mylib.step.default issue.

roger-creus commented 1 month ago

same issues here! were you able to solve them @vwxyzjn ?

I am able to run the code only if I comment this CustomOp Line. However, then the code runs at 1.8k fps with compile and cudagraphs instead of the reported 6.8k

vmoens commented 1 month ago

These should be fixed by #4 (hopefully!) LMK if it isn't!

roger-creus commented 1 month ago

@vmoens we can install and run the code now without any problems. However, I am currently unable to get the fps reported in the README. I am getting 400fps for ppo (cleanRL) and 1900 for ppo (leanRL with compile and cudagraphs).

vmoens commented 1 month ago

@vmoens we can install and run the code now without any problems. However, I am currently unable to get the fps reported in the README. I am getting 400fps for ppo (cleanRL) and 1900 for ppo (leanRL with compile and cudagraphs).

That's even a better speed up than the one reported no?

roger-creus commented 1 month ago

Not really, but something was wrong on my end. After rebooting my computers, I am able to reproduce the results in the readme (at least for the compiled+cudagraphs version). Concretely, running on a machine with 32 cores I am getting:

ppo_atari_envpool.py -- 3.4k sps ppo_atari_envpool_torchcompile.py -- 6.1k sps

As you see I am able to reproduce your results (which are awesome!) but the baseline speed is also faster on my end, probably because the number of cores I am using, which envpool can actually make good use of them. Great work! :)