Webots cannot load Numpy Python on Mac M1

[kodecreer]

Describe the Bug It says the it requires the x86 version for the Mac M1 version of Numpy. I don't want even see what Tensorflow does

Steps to Reproduce

1. Run Inverse Kinematics sample world2. Install Numpy and Ikpy
2. Run the Environment. Expected behavior. Import numpy and not cause any errors.

System

• Operating System: Mac Monterey 12
• Graphics Card: M1 Chip

Additional context The error says that the build needed is x86 and we have the ARM instead. This brings into question if this is still running x86. Would recompilation be needed?

[omichel]

Which version of Webots are you using?

The new version of Webots, R2023a, which will be released today, has a completely new Python interface which is processor agnostic. So, I would recommend to try it out and report here if you encounter any problem.

[kodecreer]

I tried the latest binary available online at the time of writing the post, but I will try that out when it's released today

[omichel]

You can get it now. It is already there.

[kodecreer]

Problem still persists in the latest version

Original error was: dlopen(/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/numpy/core/_multiarray_umath.cpython-310-darwin.so, 0x0002): tried: '/Users/name/Documents/Robot Simulations/controllers/inverse_kinematics/_multiarray_umath.cpython-310-darwin.so' (no such file), '/Applications/Webots.app/Contents/lib/controller/_multiarray_umath.cpython-310-darwin.so' (no such file), '/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/numpy/core/_multiarray_umath.cpython-310-darwin.so' **(mach-o file, but is an incompatible architecture (have (arm64), need (x86_64)))**

I also checked the version of Python I have and numpy and they are the correct versions expected.

[omichel]

Can you run a simple python program that is not a Webots controller and that uses numpy?

[kodecreer]

For sure. This is enough to reproduce the problem. import numpy as np print(np.array([1])) This is enough to cause the whole program to crash. Which I discovered that it wasn't an issue with the inverse kinematics library but more of an issue with numpy. My hypothesis is that it may have to do with the way the numpy was built in Webots and the build in the PATH python which is Arm.

[omichel]

numpy is not built in Webots. However, when initializing a Robot object in a controller program, python will load a shared library named libController.dylib which is a FAT binary with both the arm64 and x86_64 architectures. Python will likely load only one of them. It could be that the x86_64 version is loaded at this point. Then numpy will try to load a numpy shared library which only has one architecture (for example arm64) and is different from the one already loaded. This will cause the failure you observed. A solution would be that the python controller will first load the numpy, use it (causing the load of the arm64 shared library) and then initialize the Robot (causing the load of the arm64 version of the libController.dylib).

Can you try this:

import numpy as np
print(np.array([1]))

from controller import Robot, Motor
while robot.step(64) != -1:
   pass

Does it work?

[kodecreer]

Yea it still causes the same bumpy error with that code, even after shifting the placement array from where you originally mentioned.

[omichel]

I just tested it on my M1 Mac and it works fine. I guess you have a special python version with non-universal binaries. My version of Python is 3.11.0 which I downloaded from https://python.org and I installed numpy with python3 -m pip install numpy.

[kodecreer]

I just tested it on my M1 and made sure it was a universal build. It's still pulling the same error, just with a different version of python I installed.

I tried this on both python 3.11.0 and python 3.10.8 from https://www.python.org/downloads/release/python-3108/ Both are displaying the same error.

[omichel]

Are you sure Webots is using these versions of Python you just installed? When you type python3 -V from the Terminal or which python3, do you get the correct version? Note: in the Webots preferences, you can set explicitly the version of python you want to use. In order to assess you are using the correct version, you can also print it from the controller:

import sys
print(sys.version)

[omichel]

I just tested with Python 3.10.8 downloaded from https://www.python.org/ftp/python/3.10.8/python-3.10.8-macos11.pkg and it also worked out nicely. In the Webots preferences, I had to set the Python command to "python3.10" otherwise, it would still use Python 3.11 which I installed previously. I also had to install numpy this way from the Terminal:

python3.10 -m pip install numpy

I also checked that Python 3.10 is the one I just installed:

which python3.10
/Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10

[kodecreer]

which python3 outputted the following /Library/Frameworks/Python.framework/Versions/3.10/bin/python3

python3 -V output: Python 3.10.8

sys.version outputted this from the controller. 3.10.8 (v3.10.8:aaaf517424, Oct 11 2022, 10:14:40) [Clang 13.0.0 (clang-1300.0.29.30)]

[omichel]

I have the very same version as you running from the controller and numpy works fine... That is very strange... I would recommend you to completely uninstall Python 3.10 and 3.11, delete the Library/Frameworks/Python.frameworks/Version/3.10 and 3.11 folders and re-install Python 3.10 and numpy from scratch.

[kodecreer]

Here is the steps I have taken

My exact version of Mac OS is 12.5.1

• Clear the pip cache
• Deleted the Library.frameworks file and the applications file
• Check homebrew to see if any other version of python was installed
• Deleted Gazebo which had python for some reason (however it wasn't reference to at all)
• Reinstalled the python 3.10 from the link provided and reinstalled numpy

Still receiving the same results. What info would you need to give you a better diagnosis of the issue? Is there anything like logs contained in the application I can share with you?

[omichel]

My macOS version is 12.6.1. It's a Mac Mini M1 from 2020. I have no clue on why it works here and not on your system...

[kodecreer]

Alright so it seems my hunch about the OS being an issue is right. After upgrading to the same exact OS as you are using to Monterey 12.6.1 everything is working all great. Thanks for trying to help me out.

So therefore Mac OS Monterrey 12.5.1 doesn't work with WeBots

[omichel]

So therefore Mac OS Monterrey 12.5.1 doesn't work with WeBots

That's odd. My guess is rather that your system was somewhat messed-up and upgrading to Monterey 12.6.1 did some clean-up which repaired it.

[kodecreer]

Ok bad news the same issue came up again.

[omichel]

Did you install any software (e.g., some special version of Python or modules) meanwhile?

[kodecreer]

No I did not. Nothing that could influence the PATH variable. I noticed a couple of hombrew modules had python so I cleared and restart the the machine. After that still having the same issue. Even when adjusting the command for the absolute path in Webots.

[kodecreer]

You may find this funny but found out the cause of the issue.

It was opening Rosetta by default on Mac M1 which explains the CPU architecture error. Is there somewhere in the docs I can fill in to make sure that people know that it will try to run Rosetta x86 instead of ARM by default?

[omichel]

Thank you for the feedback! I wasn't expecting this. It is strange that this setting is enabled by default on some Mac and not on others... Maybe you could add a troubleshooting section in the macOS installation instructions. Please go ahead with this link to propose your contribution.