search

Farama-Foundation / D4RL

A collection of reference environments for offline reinforcement learning
Apache License 2.0
1.35k stars 285 forks source link

Cython complaint when running example python script that uses d4rl #231

Open HanYuanHsu opened 10 months ago

HanYuanHsu commented 10 months ago

When I tried to run the following python script

import gym
import d4rl # Import required to register environments, you may need to also import the submodule

# Create the environment
env = gym.make('maze2d-umaze-v1')

# d4rl abides by the OpenAI gym interface
env.reset()
env.step(env.action_space.sample())

# Each task is associated with a dataset
# dataset contains observations, actions, rewards, terminals, and infos
dataset = env.get_dataset()
print(dataset['observations']) # An N x dim_observation Numpy array of observations

I got the following error message:

Compiling /home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/cymj.pyx because it changed.
[1/1] Cythonizing /home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/cymj.pyx
performance hint: /home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/cymj.pyx:67:5: Exception check on 'c_warning_callback' will always require the GIL to be acquired.
Possible solutions:
    1. Declare 'c_warning_callback' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
    2. Use an 'int' return type on 'c_warning_callback' to allow an error code to be returned.
performance hint: /home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/cymj.pyx:104:5: Exception check on 'c_error_callback' will always require the GIL to be acquired.
Possible solutions:
    1. Declare 'c_error_callback' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
    2. Use an 'int' return type on 'c_error_callback' to allow an error code to be returned.

Error compiling Cython file:
------------------------------------------------------------
...
    See c_warning_callback, which is the C wrapper to the user defined function
    '''
    global py_warning_callback
    global mju_user_warning
    py_warning_callback = warn
    mju_user_warning = c_warning_callback
                       ^
------------------------------------------------------------

/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/cymj.pyx:92:23: Cannot assign type 'void (const char *) except * nogil' to 'void (*)(const char *) noexcept nogil'. Exception values are incompatible. Suggest adding 'noexcept' to type 'void (const char *) except * nogil'.

Error compiling Cython file:
------------------------------------------------------------
...
    See c_warning_callback, which is the C wrapper to the user defined function
    '''
    global py_error_callback
    global mju_user_error
    py_error_callback = err_callback
    mju_user_error = c_error_callback
                     ^
------------------------------------------------------------

/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/cymj.pyx:127:21: Cannot assign type 'void (const char *) except * nogil' to 'void (*)(const char *) noexcept nogil'. Exception values are incompatible. Suggest adding 'noexcept' to type 'void (const char *) except * nogil'.
Traceback (most recent call last):
  File "/home/hanyuan/Desktop/d4rl_test/test.py", line 2, in <module>
    import d4rl # Import required to register environments, you may need to also import the submodule
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/d4rl/__init__.py", line 14, in <module>
    import d4rl.locomotion
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/d4rl/locomotion/__init__.py", line 2, in <module>
    from d4rl.locomotion import ant
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/d4rl/locomotion/ant.py", line 20, in <module>
    import mujoco_py
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/__init__.py", line 2, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/builder.py", line 504, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/builder.py", line 110, in load_cython_ext
    cext_so_path = builder.build()
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/builder.py", line 226, in build
    built_so_file_path = self._build_impl()
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/builder.py", line 296, in _build_impl
    so_file_path = super()._build_impl()
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/builder.py", line 239, in _build_impl
    dist.ext_modules = cythonize([self.extension])
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1154, in cythonize
    cythonize_one(*args)
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: /home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/cymj.pyx

Does anyone know how to solve this? Here are the packages installed in my environment:

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             5.1                       1_gnu  
absl-py                   2.0.0                    pypi_0    pypi
bzip2                     1.0.8                h7b6447c_0  
ca-certificates           2023.12.12           h06a4308_0  
certifi                   2023.11.17               pypi_0    pypi
cffi                      1.16.0                   pypi_0    pypi
charset-normalizer        3.3.2                    pypi_0    pypi
click                     8.1.7                    pypi_0    pypi
cloudpickle               3.0.0                    pypi_0    pypi
cython                    3.0.7                    pypi_0    pypi
d4rl                      1.1                      pypi_0    pypi
dm-control                1.0.16                   pypi_0    pypi
dm-env                    1.6                      pypi_0    pypi
dm-tree                   0.1.8                    pypi_0    pypi
etils                     1.6.0                    pypi_0    pypi
fasteners                 0.19                     pypi_0    pypi
fsspec                    2023.12.2                pypi_0    pypi
glfw                      2.6.4                    pypi_0    pypi
gym                       0.23.1                   pypi_0    pypi
gym-notices               0.0.8                    pypi_0    pypi
h5py                      3.10.0                   pypi_0    pypi
idna                      3.6                      pypi_0    pypi
imageio                   2.33.1                   pypi_0    pypi
importlib-resources       6.1.1                    pypi_0    pypi
labmaze                   1.0.6                    pypi_0    pypi
ld_impl_linux-64          2.38                 h1181459_1  
libffi                    3.4.4                h6a678d5_0  
libgcc-ng                 11.2.0               h1234567_1  
libgomp                   11.2.0               h1234567_1  
libstdcxx-ng              11.2.0               h1234567_1  
libuuid                   1.41.5               h5eee18b_0  
lxml                      5.0.0                    pypi_0    pypi
mjrl                      1.0.0                    pypi_0    pypi
mujoco                    3.1.1                    pypi_0    pypi
mujoco-py                 2.1.2.14                 pypi_0    pypi
ncurses                   6.4                  h6a678d5_0  
numpy                     1.26.3                   pypi_0    pypi
openssl                   3.0.12               h7f8727e_0  
pillow                    10.2.0                   pypi_0    pypi
pip                       23.3.1          py310h06a4308_0  
protobuf                  4.25.1                   pypi_0    pypi
pybullet                  3.2.6                    pypi_0    pypi
pycparser                 2.21                     pypi_0    pypi
pyopengl                  3.1.7                    pypi_0    pypi
pyparsing                 3.1.1                    pypi_0    pypi
python                    3.10.13              h955ad1f_0  
readline                  8.2                  h5eee18b_0  
requests                  2.31.0                   pypi_0    pypi
scipy                     1.11.4                   pypi_0    pypi
setuptools                68.2.2          py310h06a4308_0  
sqlite                    3.41.2               h5eee18b_0  
termcolor                 2.4.0                    pypi_0    pypi
tk                        8.6.12               h1ccaba5_0  
tqdm                      4.66.1                   pypi_0    pypi
typing-extensions         4.9.0                    pypi_0    pypi
tzdata                    2023d                h04d1e81_0  
urllib3                   2.1.0                    pypi_0    pypi
wheel                     0.41.2          py310h06a4308_0  
xz                        5.4.5                h5eee18b_0  
zipp                      3.17.0                   pypi_0    pypi
zlib                      1.2.13               h5eee18b_0

I have also downloaded mujoco to the folder ~/.mujoco/mujoco210 according to the instruction.

dranaju commented 10 months ago

@HanYuanHsu, have you been able to solve this problem?

HanYuanHsu commented 10 months ago

@dranaju no, I modified the c_warning_callback function in python3.10/site-packages/mujoco_py/cymj.pyx to return an interger like the following: 

cdef int c_warning_callback(const char *msg) with gil:
    '''
    Wraps the warning callback so we can raise exceptions.
    Because callbacks can't propagate exceptions, we set a global that has
        to be inspected later.
    Use wrap_mujoco_warning() to check for that saved exception and
        re-raise it back to the caller.
    '''
    global py_warning_callback
    try:
        (<object> py_warning_callback)(msg)
    except Exception as e:
        global py_warning_exception
        py_warning_exception = e

    return -10

Same for the c_error_callback function. But then I got the following error:

Error compiling Cython file:
------------------------------------------------------------
...
    See c_warning_callback, which is the C wrapper to the user defined function
    '''
    global py_warning_callback
    global mju_user_warning
    py_warning_callback = warn
    mju_user_warning = c_warning_callback
                       ^
------------------------------------------------------------

/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/cymj.pyx:94:23: Cannot assign type 'int (const char *) except? -1 nogil' to 'void (*)(const char *) noexcept nogil'

Error compiling Cython file:
------------------------------------------------------------
...
    See c_warning_callback, which is the C wrapper to the user defined function
    '''
    global py_error_callback
    global mju_user_error
    py_error_callback = err_callback
    mju_user_error = c_error_callback
                     ^
------------------------------------------------------------

/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/cymj.pyx:131:21: Cannot assign type 'int (const char *) except? -1 nogil' to 'void (*)(const char *) noexcept nogil'
Traceback (most recent call last):
  File "/home/hanyuan/Desktop/d4rl_test/test.py", line 2, in <module>
    import d4rl # Import required to register environments, you may need to also import the submodule
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/d4rl/__init__.py", line 14, in <module>
    import d4rl.locomotion
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/d4rl/locomotion/__init__.py", line 2, in <module>
    from d4rl.locomotion import ant
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/d4rl/locomotion/ant.py", line 20, in <module>
    import mujoco_py
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/__init__.py", line 2, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/builder.py", line 504, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/builder.py", line 110, in load_cython_ext
    cext_so_path = builder.build()
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/builder.py", line 226, in build
    built_so_file_path = self._build_impl()
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/builder.py", line 296, in _build_impl
    so_file_path = super()._build_impl()
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/builder.py", line 239, in _build_impl
    dist.ext_modules = cythonize([self.extension])
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1154, in cythonize
    cythonize_one(*args)
  File "/home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: /home/hanyuan/anaconda3/envs/d4rl/lib/python3.10/site-packages/mujoco_py/cymj.pyx
dranaju commented 10 months ago

@HanYuanHsu, I've successfully managed to run this part by downgrading the Cython version to less than 3. My current Cython version is 0.29.37. To do the same, you can simply execute the command pip install 'cython<3'.