search

tensorflow / agents

TF-Agents: A reliable, scalable and easy to use TensorFlow library for Contextual Bandits and Reinforcement Learning.
Apache License 2.0
2.77k stars 714 forks source link

tf-agents ubuntu 24.04 venv pip3 install error on pygame 2.1.3 #933

Open johnny-littlepunch opened 1 month ago

johnny-littlepunch commented 1 month ago

(pyenv) dev@ud24:~$ pip3 install tf-agents Collecting tf-agents Using cached tf_agents-0.19.0-py3-none-any.whl.metadata (12 kB) Collecting absl-py>=0.6.1 (from tf-agents) Using cached absl_py-2.1.0-py3-none-any.whl.metadata (2.3 kB) Collecting cloudpickle>=1.3 (from tf-agents) Using cached cloudpickle-3.0.0-py3-none-any.whl.metadata (7.0 kB) Collecting gin-config>=0.4.0 (from tf-agents) Using cached gin_config-0.5.0-py3-none-any.whl.metadata (2.9 kB) Collecting gym<=0.23.0,>=0.17.0 (from tf-agents) Using cached gym-0.23.0.tar.gz (624 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting numpy>=1.19.0 (from tf-agents) Using cached numpy-2.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB) Collecting pillow (from tf-agents) Using cached pillow-10.4.0-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (9.2 kB) Collecting six>=1.10.0 (from tf-agents) Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) Collecting protobuf>=3.11.3 (from tf-agents) Using cached protobuf-5.27.2-cp38-abi3-manylinux2014_x86_64.whl.metadata (592 bytes) Collecting wrapt>=1.11.1 (from tf-agents) Using cached wrapt-1.16.0-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB) Collecting typing-extensions==4.5.0 (from tf-agents) Using cached typing_extensions-4.5.0-py3-none-any.whl.metadata (8.5 kB) Collecting pygame==2.1.3 (from tf-agents) Using cached pygame-2.1.3.tar.gz (12.8 MB) Installing build dependencies ... done Getting requirements to build wheel ... error error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> [47 lines of output]

  WARNING, No "Setup" File Exists, Running "buildconfig/config.py"
  Using UNIX configuration...

  /bin/sh: 1: sdl2-config: not found
  /bin/sh: 1: sdl2-config: not found
  /bin/sh: 1: sdl2-config: not found

  Hunting dependencies...

  ---
  For help with compilation see:
      https://www.pygame.org/wiki/Compilation
  To contribute to pygame development see:
      https://www.pygame.org/contribute.html
  ---

  Traceback (most recent call last):
    File "/home/dev/Documents/pyenv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/home/dev/Documents/pyenv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/dev/Documents/pyenv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
             ^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-3lrzlwlv/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=[])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-3lrzlwlv/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-3lrzlwlv/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 497, in run_setup
      super().run_setup(setup_script=setup_script)
    File "/tmp/pip-build-env-3lrzlwlv/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 313, in run_setup
      exec(code, locals())
    File "<string>", line 400, in <module>
    File "/tmp/pip-install-3m5hgfi8/pygame_88b1a3367ab641ed8187679a907878c8/buildconfig/config.py", line 231, in main
      deps = CFG.main(**kwds, auto_config=auto)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-install-3m5hgfi8/pygame_88b1a3367ab641ed8187679a907878c8/buildconfig/config_unix.py", line 218, in main
      DependencyProg('SDL', 'SDL_CONFIG', 'sdl2-config', '2.0', ['sdl']),
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-install-3m5hgfi8/pygame_88b1a3367ab641ed8187679a907878c8/buildconfig/config_unix.py", line 43, in __init__
      self.ver = config[0].strip()
                 ~~~~~~^^^
  IndexError: list index out of range
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

drakon commented 1 month ago

I have the same issue that I don't get tf-agents to install properly. 

(venv) root@a7f070a4b7cb:/# pip install tf-agents[reverb]==0.19.0
Requirement already satisfied: tf-agents==0.19.0 in /venv/lib/python3.10/site-packages (from tf-agents[reverb]==0.19.0) (0.19.0)
Requirement already satisfied: absl-py>=0.6.1 in /venv/lib/python3.10/site-packages (from tf-agents==0.19.0->tf-agents[reverb]==0.19.0) (2.1.0)
Requirement already satisfied: cloudpickle>=1.3 in /venv/lib/python3.10/site-packages (from tf-agents==0.19.0->tf-agents[reverb]==0.19.0) (3.0.0)
Requirement already satisfied: gin-config>=0.4.0 in /venv/lib/python3.10/site-packages (from tf-agents==0.19.0->tf-agents[reverb]==0.19.0) (0.5.0)
Requirement already satisfied: gym<=0.23.0,>=0.17.0 in /venv/lib/python3.10/site-packages (from tf-agents==0.19.0->tf-agents[reverb]==0.19.0) (0.23.0)
Requirement already satisfied: numpy>=1.19.0 in /venv/lib/python3.10/site-packages (from tf-agents==0.19.0->tf-agents[reverb]==0.19.0) (2.0.1)
Requirement already satisfied: pillow in /venv/lib/python3.10/site-packages (from tf-agents==0.19.0->tf-agents[reverb]==0.19.0) (10.4.0)
Requirement already satisfied: six>=1.10.0 in /venv/lib/python3.10/site-packages (from tf-agents==0.19.0->tf-agents[reverb]==0.19.0) (1.16.0)
Requirement already satisfied: protobuf>=3.11.3 in /venv/lib/python3.10/site-packages (from tf-agents==0.19.0->tf-agents[reverb]==0.19.0) (5.27.3)
Requirement already satisfied: wrapt>=1.11.1 in /venv/lib/python3.10/site-packages (from tf-agents==0.19.0->tf-agents[reverb]==0.19.0) (1.16.0)
Requirement already satisfied: typing-extensions==4.5.0 in /venv/lib/python3.10/site-packages (from tf-agents==0.19.0->tf-agents[reverb]==0.19.0) (4.5.0)
Requirement already satisfied: pygame==2.1.3 in /venv/lib/python3.10/site-packages (from tf-agents==0.19.0->tf-agents[reverb]==0.19.0) (2.1.3)
Requirement already satisfied: tensorflow-probability~=0.23.0 in /venv/lib/python3.10/site-packages (from tf-agents==0.19.0->tf-agents[reverb]==0.19.0) (0.23.0)
INFO: pip is looking at multiple versions of tf-agents[reverb] to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement rlds; extra == "reverb" (from tf-agents[reverb]) (from versions: none)
ERROR: No matching distribution found for rlds; extra == "reverb"

If I don't specify the version, it will search and ends up installing 0.5.0 and then get this warning (and subsequently it doesn't find the import reverb):

WARNING: tf-agents 0.5.0 does not provide the extra 'reverb'

I tried to install both dm-reverb and rlds directly but with both I get:

ERROR: Could not find a version that satisfies the requirement dm-reverb (from versions: none)

I tried it both with docker python (debian) and ubuntu (and installed python myself). Both for python 3.11 and 3.10 with the same results.

Is there anything to be aware of to get this installed properly? Does Docker just not work?

PS: I remember also getting the above error at some point, but don't know anymore under which circumstances.

drakon commented 1 month ago

Ok, I'm not sure if the problem the question is related to is based on this but in regards to Docker on MacOS, I found out the following.

  1. The binaries for tf-agents (and just tensorflow as well) and in particular dm-reverb not only work only under Linux OS but also only for linux/amd64 architecture (so it doesn't run under arm64/Apple Silicon based).
  2. The above issues I posted are based on pypi trying to find a binary that works on arm64 (which there are none, except tf-agents 0.5.0?)
  3. I tried a simple Docker (python:3.11 based with pip install tf-agents[reverb]) and it works under Windows Docker but not MacOS (with --platform linux/amd64 flag)
  4. Under MacOS I get "Illegal instruction" when importing tensorflow, tf_agents or reverb) when Docker is using Rosetta. This is a known issue of Docker/Rosetta
  5. Disabling Rosetta in Docker MacOS then leads the Dockerfile working (but it seems quite a bit slower)