openai / mujoco-py

MuJoCo is a physics engine for detailed, efficient rigid body simulations with contacts. mujoco-py allows using MuJoCo from Python 3.
Other
2.87k stars 815 forks source link

mujoco_py Does not Recognize GCC@10 #605

Open DarrenZhang01 opened 3 years ago

DarrenZhang01 commented 3 years ago

Describe the bug As titled. For Mac with M1 chips, it is impossible to use older versions of GCC, as described in https://formulae.brew.sh/formula/gcc@6. What are the possible workarounds? Thanks.

To Reproduce Install mujoco-py with GCC@10, and you will see the error.

Expected behavior How to make mujoco-py recognize GCC@10

Error Messages

RuntimeError: Could not find GCC executable.

Desktop (please complete the following information):

josuenassar commented 3 years ago

I'm also getting a similar issue trying to run some code that uses mujoco-py on apple m1. When I try running I get the following error distutils.errors.CompileError: command '/usr/local/bin/gcc-8' failed with exit status 1

drozzy commented 3 years ago

Macbook air m1 2021 (after installing gcc with brew install gcc):

 RuntimeError: Could not find GCC executable.

  HINT: On OS X, install GCC with `brew install gcc`. or `port install gcc`.
  ----------------------------------------
  ERROR: Failed building wheel for mujoco-py
wookayin commented 3 years ago

The gcc paths are hardcoded in the builder script, unfortunately. Do the authors even care future compatibility? The script supports up to gcc 9. However, unfortunately, gcc@9 cannot be installed on arm64 machines.

A dirty, hacky workaround is to create as a symbolic link as follows:

ln -sf /opt/homebrew/bin/gcc-11 /usr/local/bin/gcc-9

Yes, I know this is totally a nonsense but with this gcc can be detected. Or alternatively, you could set an environment variable like export CC=/opt/homebrew/bin/gcc-11 and then do pip install.

However, I failed to successfully the extension due to the architecture difference, running into the following error:

.../mujoco_py/generated/cymj_2.0.2.13_39_macextensionbuilder_39.so, 2): no suitable image found.  Did find:
        /private/var/folders/yj/.../T/pip-install-as50_4qo/mujoco-py_e1a..../mujoco_py/generated/cymj_2.0.2.13_39_macextensionbuilder_39.so: mach-o, but wrong architecture

Dear OpenAI, please maintain the package!

wookayin commented 3 years ago

I tried a couple of things. In short, I don't think one can run mujoco on a M1 arm_64 machine.

First, the mujoco on macos is released with x86_64 binary, so we have to use x86_64 for python and cython extensions. In order to do so:

Install gcc using "x86_64" homebrew: This can be typically done via /usr/local/bin/brew install gcc@9. Note that this should be a different homebrew installation than arm64 brew (/opt/homebrew/bin/...) You will see gcc is installed at /usr/local/bin/gcc-9.

Make sure that you are using x86_64 python. You can verify this with the command lipo -archs $(which python)

Export CC=/usr/local/bin/gcc-9 to force compiler detection to x86_64-compatible gcc before installing mujoco-py. The resulting shared module mujoco_py/generated/cymj_2.0.2.13_38_macextensionbuilder_38.so will be a x86_64 binary rather than arm64.

However, the mujoco binary itself uses a x86_64 specific hardware instructions (AVX2) which cannot be emulated using rosetta2. So my conclusion is that until mujoco officially supports M1, there is no chance one could run mujoco-py (and raw mujoco itself) on arm64 mac.

[1]    42661 illegal hardware instruction  python setup.py build
ConnerQiu commented 3 years ago

I soloved the gcc problem on Macbook pro with m1 chip. here are the steps I did:

  1. enter the downloaded mujoco folder from github, hand install mujoco.py by running 'python setup.py install' in terminal
  2. found the builder.py in your env package folder, the path in my device is ‘/Users/xx/miniforge3/envs/dl/lib/python3.9/site-packages/
  3. in line 321 of builder.py, add your gcc-11 filepath to the ‘_build_impl' function as follows:

        c_compilers = [
            '/usr/local/bin/gcc-9',
            '/usr/local/bin/gcc-8',
            '/usr/local/bin/gcc-7',
            '/usr/local/bin/gcc-6',
            '/opt/local/bin/gcc-mp-9',
            '/opt/local/bin/gcc-mp-8',
            '/opt/local/bin/gcc-mp-7',
            '/opt/local/bin/gcc-mp-6',
            '/opt/homebrew/Cellar/gcc/11.1.0_1/bin/gcc-11'
        ]
  4. if you could not find your gcc-11 path, try 'brew reinstall gcc', it will tell you where the gcc was installed.

However, after doing this, i still got a import error:

ImportError: dlopen(/Users/cq/miniforge3/envs/dl/lib/python3.9/site-packages/mujoco_py-2.0.2.13-py3.9.egg/mujoco_py/generated/cymj_2.0.2.13_39_macextensionbuilder_39.so, 2): Symbol not found: _mjcb_act_bias Referenced from: /Users/cq/miniforge3/envs/dl/lib/python3.9/site-packages/mujoco_py-2.0.2.13-py3.9.egg/mujoco_py/generated/cymj_2.0.2.13_39_macextensionbuilder_39.so Expected in: flat namespace in /Users/cq/miniforge3/envs/dl/lib/python3.9/site-packages/mujoco_py-2.0.2.13-py3.9.egg/mujoco_py/generated/cymj_2.0.2.13_39_macextensionbuilder_39.so

hope these information helps.

yorgoon commented 3 years ago

I soloved the gcc problem on Macbook pro with m1 chip. here are the steps I did:

  1. enter the downloaded mujoco folder from github, hand install mujoco.py by running 'python setup.py install' in terminal
  2. found the builder.py in your env package folder, the path in my device is ‘/Users/xx/miniforge3/envs/dl/lib/python3.9/site-packages/
  3. in line 321 of builder.py, add your gcc-11 filepath to the ‘_build_impl' function as follows:
        c_compilers = [
            '/usr/local/bin/gcc-9',
            '/usr/local/bin/gcc-8',
            '/usr/local/bin/gcc-7',
            '/usr/local/bin/gcc-6',
            '/opt/local/bin/gcc-mp-9',
            '/opt/local/bin/gcc-mp-8',
            '/opt/local/bin/gcc-mp-7',
            '/opt/local/bin/gcc-mp-6',
            '/opt/homebrew/Cellar/gcc/11.1.0_1/bin/gcc-11'
        ]
  4. if you could not find your gcc-11 path, try 'brew reinstall gcc', it will tell you where the gcc was installed.

However, after doing this, i still got a import error:

ImportError: dlopen(/Users/cq/miniforge3/envs/dl/lib/python3.9/site-packages/mujoco_py-2.0.2.13-py3.9.egg/mujoco_py/generated/cymj_2.0.2.13_39_macextensionbuilder_39.so, 2): Symbol not found: _mjcb_act_bias Referenced from: /Users/cq/miniforge3/envs/dl/lib/python3.9/site-packages/mujoco_py-2.0.2.13-py3.9.egg/mujoco_py/generated/cymj_2.0.2.13_39_macextensionbuilder_39.so Expected in: flat namespace in /Users/cq/miniforge3/envs/dl/lib/python3.9/site-packages/mujoco_py-2.0.2.13-py3.9.egg/mujoco_py/generated/cymj_2.0.2.13_39_macextensionbuilder_39.so

hope these information helps.

Same here.

ld: warning: ignoring file /Users/soowonkim/.mujoco/mujoco200/bin/libmujoco200.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64 ld: warning: ignoring file /Users/soowonkim/.mujoco/mujoco200/bin/libglfw.3.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64 Traceback (most recent call last): File "", line 1, in File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/mujoco_py-2.0.2.13-py3.9.egg/mujoco_py/init.py", line 3, in from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/mujoco_py-2.0.2.13-py3.9.egg/mujoco_py/builder.py", line 511, in cymj = load_cython_ext(mujoco_path) File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/mujoco_py-2.0.2.13-py3.9.egg/mujoco_py/builder.py", line 106, in load_cython_ext mod = load_dynamic_ext('cymj', cext_so_path) File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/mujoco_py-2.0.2.13-py3.9.egg/mujoco_py/builder.py", line 125, in load_dynamic_ext return loader.load_module() ImportError: dlopen(/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/mujoco_py-2.0.2.13-py3.9.egg/mujoco_py/generated/cymj_2.0.2.13_39_macextensionbuilder_39.so, 2): Symbol not found: _mjcb_act_bias Referenced from: /opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/mujoco_py-2.0.2.13-py3.9.egg/mujoco_py/generated/cymj_2.0.2.13_39_macextensionbuilder_39.so Expected in: flat namespace in /opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/mujoco_py-2.0.2.13-py3.9.egg/mujoco_py/generated/cymj_2.0.2.13_39_macextensionbuilder_39.so

wookayin commented 3 years ago

@ConnerQiu As I mentioned, even though you succeeded installing the mujoco-py, you cannot run mujoco on M1 because mujoco binary does not support M1/arm64 architecture.

ConnerQiu commented 3 years ago

@ConnerQiu As I mentioned, even though you succeeded installing the mujoco-py, you cannot run mujoco on M1 because mujoco binary does not support M1/arm64 architecture.

Yes, I agree. we could only wait a newly released arm64 version.

qazi0 commented 2 years ago

So we still don't have mujoco-py working on M1? :(

VijayVignesh1 commented 1 year ago

Is it still not supported on M1?

JuliusJacobitz commented 11 months ago

Any news?

wookayin commented 11 months ago

The project (mujoco-py) is abandoned and no longer maintained. Your best bet is to use the workaround I posted or move to the mujoco python binding https://github.com/google-deepmind/mujoco/ ASAP.