apple / tensorflow_macos

TensorFlow for macOS 11.0+ accelerated using Apple's ML Compute framework.
Other
3.66k stars 308 forks source link

Tensorflow cannot be installed in Mac M1 because of error ERROR: numpy-1.18.5-cp38-cp38-macosx_11_0_arm64.whl is not a supported wheel on this platform. #48

Open monikavila opened 3 years ago

monikavila commented 3 years ago

I am trying to install tensor flow in the new MacBook Pro M1 but it gives the error ERROR: numpy-1.18.5-cp38-cp38-macosx_11_0_arm64.whl is not a supported wheel on this platform.

Aizen741 commented 3 years ago

@jrlindell

Dont create new Virtual Environment under Anaconda. You have to create it under Miniforge

First, get out of the anaconda env , and after you download the miniforge create the Virtual Environment under Miniforge .

Screenshot 2021-03-06 at 10 56 16 AM

Once you create Env in Miniforge this will come . So Don't Close this terminal yet. open a new terminal

Screenshot 2021-03-06 at 11 11 31 AM

Download Additional file

Screenshot 2021-03-06 at 11 05 51 AM

Downloading the Tensorflow : Download tensorflow from releases , Once you download and unzip it open the file and drag - drop install_venv into terminal under the env also add -p in the end. it will ask for the location . provide the same location as the env.

Screenshot 2021-03-06 at 11 15 16 AM

There You go . Tensorflow is working for me perfectly on m1 even in jupyter notebook too

Screenshot 2021-03-06 at 11 19 37 AM
karisjochen commented 3 years ago

I am following this tutorial: https://github.com/apple/tensorflow_macos/issues/153 and have downloaded miniforge3 but the conda command is only returning: "Killed: 9". I have tried to add the miniforge to my path. I have restarted the terminal and I have restarted my computer. Does anyone have any ideas as to what the issue may be?

mwidjaja1 commented 3 years ago

@karisjochen could you please paste your outputs when you try running the Sanity Checks and Troubleshooting sections? I'm willing to bet you missed a sanity check along the way (which I'm willing to bet since one of the bullet points have very specific instructions on how to ask for help)

karisjochen commented 3 years ago

Here are the first two sanity checks. I have not progressed to actually downloading TensorFlow since my conda command does not work.

image image image
mwidjaja1 commented 3 years ago

What happens when you run file $(which conda)?

EDIT: I'm an idiot this won't work. Don't try this.

mwidjaja1 commented 3 years ago

Try running conda init with a -v command thrown in to see if anything more gets printed.

Killed: 9 is an indicator that your computer chose to kill conda. (https://stackoverflow.com/questions/16338884/what-does-exited-abnormally-with-signal-9-killed-9-mean). It's possible you have software or management configurations that prohibit you from using conda. It's also possible that you are not using a shell that conda is programmed to use (you must use {bash, fish, powershell, tcsh, xonsh, or zsh}), and because of that, your computer kills it. Another debugging step would be to trash miniforge3 and reinstall it by downloading a new installer.

After that I think I'm all out of ideas. This appears to be a computer configuration issue, I'm not aware of anything inside of Conda itself that can cause this issue.

karisjochen commented 3 years ago

"conda init -v" yields the same "killed: 9" response. I am using bash so that shouldn't be an issue. In your user directory where you have miniforge, can you show a screenshot or list your hidden directories that deal with conda? I am wondering if this is a computer configuration issue and something got deleted when someone was trying to clean up my computer.

mwidjaja1 commented 3 years ago

So a lot of the stuff miniforge3 (and indirectly, Homebrew) uses are often in directories that seem like they can't possibly be legit... but they are. So that does not surprise me at all. There's nothing special in my miniforge3 directory -- only a lone .condarc as an invisible file.

Still, reinstalling miniforge would fix any issues if that was a problem. Miniforge is contained to itself in its directory.

karisjochen commented 3 years ago

yes I can see miniforge as a directory in my home directory. And it has the same .condarc file. I reinstalled it yet i am getting the same killed:9 error. I dont know where to go from here.

karisjochen commented 3 years ago

FYI in case anyone also struggles with this issue. I had to call apple support and disable some privacy settings on my computer for the conda commands to work but now I was able to download both miniforge and tensorflow so life is good!

SmithWinter commented 3 years ago

hello guys I think I have found the solution. first, uninstall anaconda and any python3 that you didn't download from apple. then please xcode-select --install(maybe you can download Xcode from app store) after that, you will get python3.8.2 from apple , please check if your python is 3.8.2 and it's path is/usr/bin/python3. if succeed, run the install_venv.sh again. ps:in the last step, you should build a new venv,or just delete the old venv on anaconda/python3 not downloaded from apple and it works finally.

it's work too, thanks you so much

MaxKumundzhiev commented 3 years ago

Additional tutorial which could help to setup TF within M1 approaching by miniforge here.

gabirelasanchezzz commented 3 years ago

I got Tensor Flow working on my Anaconda Spyder.

As I was a noob to these, so let me share my detailed findings for other people in similar boat.

First I downloaded: https://github.com/apple/tensorflow_macos/releases/download/v0.1alpha0/tensorflow_macos-0.1alpha0.tar.gz

After unzipping, there is 2 folders inside arm64 and x86_64. The first reaction is for people to install arm64, since we have the Mac M1 chip. But actually since we wanted to install into Anaconda which runs through Rosetta2 as an Intel, we should install x86_64, which will run slower, but at least you have all the libraries such as pandas and support apps.

To Install x86_64 into Anaconda (so I can run in Spyder):

  1. Anaconda start a new environment
  2. Uninstall tensorflow & tensorboard if exists
  3. Rename all the files inside the x86_64 folder from _110 to _1016. Example: tensorflow_macos-0.1a0-cp38-cp38-macosx_10_16_x86_64.whl
  4. In Conda console run below with 'username' replaced with your username: pip install pip wheel setuptools cached-property six pip install "/Users/username/Downloads/tensorflow_macos/x86_64/grpcio-1.33.2-cp38-cp38-macosx_10_16_x86_64.whl" pip install "/Users/username/Downloads/tensorflow_macos/x86_64/h5py-2.10.0-cp38-cp38-macosx_10_16_x86_64.whl" pip install "/Users/username/Downloads/tensorflow_macos/x86_64/numpy-1.18.5-cp38-cp38-macosx_10_16_x86_64.whl" pip install "/Users/username/Downloads/tensorflow_macos/x86_64/scipy-1.5.4-cp38-cp38-macosx_10_16_x86_64.whl" pip install "/Users/username/Downloads/tensorflow_macos/x86_64/tensorflow_addons-0.11.2+mlcompute-cp38-cp38-macosx_10_16_x86_64.whl" pip install absl-py astunparse flatbuffers gast google_pasta keras_preprocessing opt_einsum protobuf tensorflow_estimator termcolor typing_extensions wrapt wheel tensorboard typeguard pip install "/Users/username/Downloads/tensorflow_macos/x86_64/tensorflow_macos-0.1a0-cp38-cp38-macosx_10_16_x86_64.whl"

Source: #7 (comment)

To Install arm64 into virtualenv This would give the fastest speed since it uses the native arm64, but it doesn't support many libraries at the moment, including pandas.

To install:

  1. With tensorflow_macos unzipped in the Downloads folder
  2. In Terminal: browse to Downloads folder
  3. Run: /bin/bash ./tensorflow_macos/install_venv.sh --prompt
  4. Key in your desired path or just Enter for the default path

To Run:

  1. In a new terminal run: source /Users/username/tensorflow_macos_venv/bin/activate
  2. python3 "your_python_file.py"

To toggle between CPU and GPU runs adjust the device_name below: from tensorflow.python.compiler.mlcompute import mlcompute mlcompute.set_mlc_device(device_name='gpu')

To make it faster, turn off the default Eager mode with the following line: tf.compat.v1.disable_eager_execution()

Run Speed Comparison for MNIST character recognition: Anaconda Spyder (x86_64, EagerOff): 200-256us/sample Terminal (arm64, CPU, EagerOff): 22-63us/sample Terminal (arm64, GPU, EagerOff): 137-166us/sample

Hi, it worked perfectly after days struggling. However I am a noob, and when I try to run it in Spyder it says

ModuleNotFoundError: No module named 'tensorflow'

But if I run: pip list | grep tensorflow it outputs: tensorflow-estimator 2.4.0 Note: you may need to restart the kernel to use updated packages.

I have restarted the kernel but nothing will work. How could I solve this?

mwidjaja1 commented 3 years ago

Any reason you aren't using the psuedo-official instructions that the Apple TensorFlow Team (and I, as a volunteer) wrote in Issue 153? https://github.com/apple/tensorflow_macos/issues/153.

Anaconda doesn't work on ARM64 -- it works in a compiled Rosetta x86 mode, but you will lose all the ARM optimizations then.

monikavila commented 3 years ago

I got Tensor Flow working on my Anaconda Spyder. As I was a noob to these, so let me share my detailed findings for other people in similar boat. First I downloaded: https://github.com/apple/tensorflow_macos/releases/download/v0.1alpha0/tensorflow_macos-0.1alpha0.tar.gz After unzipping, there is 2 folders inside arm64 and x86_64. The first reaction is for people to install arm64, since we have the Mac M1 chip. But actually since we wanted to install into Anaconda which runs through Rosetta2 as an Intel, we should install x86_64, which will run slower, but at least you have all the libraries such as pandas and support apps. To Install x86_64 into Anaconda (so I can run in Spyder):

  1. Anaconda start a new environment
  2. Uninstall tensorflow & tensorboard if exists
  3. Rename all the files inside the x86_64 folder from _110 to _1016. Example: tensorflow_macos-0.1a0-cp38-cp38-macosx_10_16_x86_64.whl
  4. In Conda console run below with 'username' replaced with your username: pip install pip wheel setuptools cached-property six pip install "/Users/username/Downloads/tensorflow_macos/x86_64/grpcio-1.33.2-cp38-cp38-macosx_10_16_x86_64.whl" pip install "/Users/username/Downloads/tensorflow_macos/x86_64/h5py-2.10.0-cp38-cp38-macosx_10_16_x86_64.whl" pip install "/Users/username/Downloads/tensorflow_macos/x86_64/numpy-1.18.5-cp38-cp38-macosx_10_16_x86_64.whl" pip install "/Users/username/Downloads/tensorflow_macos/x86_64/scipy-1.5.4-cp38-cp38-macosx_10_16_x86_64.whl" pip install "/Users/username/Downloads/tensorflow_macos/x86_64/tensorflow_addons-0.11.2+mlcompute-cp38-cp38-macosx_10_16_x86_64.whl" pip install absl-py astunparse flatbuffers gast google_pasta keras_preprocessing opt_einsum protobuf tensorflow_estimator termcolor typing_extensions wrapt wheel tensorboard typeguard pip install "/Users/username/Downloads/tensorflow_macos/x86_64/tensorflow_macos-0.1a0-cp38-cp38-macosx_10_16_x86_64.whl"

Source: #7 (comment) To Install arm64 into virtualenv This would give the fastest speed since it uses the native arm64, but it doesn't support many libraries at the moment, including pandas. To install:

  1. With tensorflow_macos unzipped in the Downloads folder
  2. In Terminal: browse to Downloads folder
  3. Run: /bin/bash ./tensorflow_macos/install_venv.sh --prompt
  4. Key in your desired path or just Enter for the default path

To Run:

  1. In a new terminal run: source /Users/username/tensorflow_macos_venv/bin/activate
  2. python3 "your_python_file.py"

To toggle between CPU and GPU runs adjust the device_name below: from tensorflow.python.compiler.mlcompute import mlcompute mlcompute.set_mlc_device(device_name='gpu') To make it faster, turn off the default Eager mode with the following line: tf.compat.v1.disable_eager_execution() Run Speed Comparison for MNIST character recognition: Anaconda Spyder (x86_64, EagerOff): 200-256us/sample Terminal (arm64, CPU, EagerOff): 22-63us/sample Terminal (arm64, GPU, EagerOff): 137-166us/sample

Hi, it worked perfectly after days struggling. However I am a noob, and when I try to run it in Spyder it says

ModuleNotFoundError: No module named 'tensorflow'

But if I run: pip list | grep tensorflow it outputs: tensorflow-estimator 2.4.0 Note: you may need to restart the kernel to use updated packages.

I have restarted the kernel but nothing will work. How could I solve this?

My solution was to install all with coda mini forge.

Another option is to install Visual Code and they released a version that could run with ARM. But my problem is that sometimes (almost always) the kernel dies. That is something I could not solve yet.

gabirelasanchezzz commented 3 years ago

Any reason you aren't using the psuedo-official instructions that the Apple TensorFlow Team (and I, as a volunteer) wrote in Issue 153? #153.

Anaconda doesn't work on ARM64 -- it works in a compiled Rosetta x86 mode, but you will lose all the ARM optimizations then.

Hi, so I have as well tried but I get the following error after I run step 3: environment variables: CIO_TEST=<not set> CONDA_AUTO_UPDATE_CONDA=false CONDA_DEFAULT_ENV=base ...

mwidjaja1 commented 3 years ago

@gabirelasanchezzz And you are sure you ran the 'Sanity Check' steps in the 'Install Miniforge' section and all of them looked right, pointing to miniforge? It really sounds like you did not install Miniforge correctly.

gabirelasanchezzz commented 3 years ago

@mwidjaja1 I have done all the steps, but I realized my after doing the sanity check I get: /.../anaconda3/bin/python: Mach-O 64-bit executable x86_64 Despite I have an M1 Mac. I have removed Miniforge and reinstalled it ensuring it is the one for arm64 architecture but still outputs the same 😭

mwidjaja1 commented 3 years ago

@gabirelasanchezzz I think you must have confused your Mac to what's going on because the guide doesn't tell you to install Anaconda and yet you have it. I dunno how much any of us can help you at this point to be honest -- this seems like a configuration issue for your Mac specifically now.

My best advice is to find your .bashrc/.zshrc file (depending on what you store all your environment variables in), delete anything related to anaconda/miniforge, delete anaconda/miniforge from your computer completely, and start again. Short of that, there's not much I think we can help you with short of starting completely from fresh.

gabirelasanchezzz commented 3 years ago

@mwidjaja1 I see. I am new here so I think I just messed the configuration. Thanks for trying to help, really!

mwidjaja1 commented 3 years ago

@gabirelasanchezzz yeah sorry 😔. I just am out of ideas on how to help you remotely unfortunately. But yeah my best advice is do your darnest to start from fresh again in regards to your terminal environment and uninstalling everything but the system Python and starting from fresh. Or create a new user account on your Mac and try setting it up there, just so at least you have an 'answer key' you can compare your currrent user account with to see where you might have went wrong.

Best of luck!

gabirelasanchezzz commented 3 years ago

@mwidjaja1 Oh I had not thought about the user. Thanks a lot!

leoloman commented 3 years ago

@jrlindell

Dont create new Virtual Environment under Anaconda. You have to create it under Miniforge

First, get out of the anaconda env , and after you download the miniforge create the Virtual Environment under Miniforge .

Screenshot 2021-03-06 at 10 56 16 AM

Once you create Env in Miniforge this will come . So Don't Close this terminal yet. open a new terminal

Screenshot 2021-03-06 at 11 11 31 AM

Download Additional file

Screenshot 2021-03-06 at 11 05 51 AM

Downloading the Tensorflow : Download tensorflow from releases , Once you download and unzip it open the file and drag - drop install_venv into terminal under the env also add -p in the end. it will ask for the location . provide the same location as the env.

Screenshot 2021-03-06 at 11 15 16 AM

There You go . Tensorflow is working for me perfectly on m1 even in jupyter notebook too

Screenshot 2021-03-06 at 11 19 37 AM

This worked perfectly for me if anyone is having trouble and has the mini forge arm installed

testforme0 commented 3 years ago

also struggles with

Do you remember what changes were made, I also encountered the same problem?

gabirelasanchezzz commented 3 years ago

also struggles with

Do you remember what changes were made, I also encountered the same problem?

Hi, I desisted. All I tried did not work. However, I have not tried yet the solution of @leoloman . I definitely need to try this. Best of luck

testforme0 commented 3 years ago

also struggles with

Do you remember what changes were made, I also encountered the same problem?

Hi, I desisted. All I tried did not work. However, I have not tried yet the solution of @leoloman . I definitely need to try this. Best of luck

I use bash instead of zsh, command success!I dont know why,but it works

may3rd commented 3 years ago

Has anyone successfully install object detection on python? I've tried without any luck. There is too much error and warning.

Any hint?

arjanvanham commented 3 years ago

I followed the Clayton Pilat's instructions from here and got everything installed that I was struggling with for weeks:

% python
Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 15:50:57) 
[Clang 11.0.1 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> import PIL
>>> import matplotlib
>>> import keras
>>> 

Apple MacBook Pro M1 / Big Sur 11.2.3, Python 3.8.8

VeronicaCPerez commented 3 years ago

I am trying to install tensor flow in the new MacBook Pro M1 but it gives the error ERROR: numpy-1.18.5-cp38-cp38-macosx_11_0_arm64.whl is not a supported wheel on this platform.

@monikavila I fixed it by downgrading my python from 3.9.1 to 3.8.6

denalist commented 3 years ago

I got Tensor Flow working on my Anaconda Spyder.

As I was a noob to these, so let me share my detailed findings for other people in similar boat.

First I downloaded: https://github.com/apple/tensorflow_macos/releases/download/v0.1alpha0/tensorflow_macos-0.1alpha0.tar.gz

After unzipping, there is 2 folders inside arm64 and x86_64. The first reaction is for people to install arm64, since we have the Mac M1 chip. But actually since we wanted to install into Anaconda which runs through Rosetta2 as an Intel, we should install x86_64, which will run slower, but at least you have all the libraries such as pandas and support apps.

To Install x86_64 into Anaconda (so I can run in Spyder):

  1. Anaconda start a new environment
  2. Uninstall tensorflow & tensorboard if exists
  3. Rename all the files inside the x86_64 folder from _110 to _1016. Example: tensorflow_macos-0.1a0-cp38-cp38-macosx_10_16_x86_64.whl
  4. In Conda console run below with 'username' replaced with your username: pip install pip wheel setuptools cached-property six pip install "/Users/username/Downloads/tensorflow_macos/x86_64/grpcio-1.33.2-cp38-cp38-macosx_10_16_x86_64.whl" pip install "/Users/username/Downloads/tensorflow_macos/x86_64/h5py-2.10.0-cp38-cp38-macosx_10_16_x86_64.whl" pip install "/Users/username/Downloads/tensorflow_macos/x86_64/numpy-1.18.5-cp38-cp38-macosx_10_16_x86_64.whl" pip install "/Users/username/Downloads/tensorflow_macos/x86_64/scipy-1.5.4-cp38-cp38-macosx_10_16_x86_64.whl" pip install "/Users/username/Downloads/tensorflow_macos/x86_64/tensorflow_addons-0.11.2+mlcompute-cp38-cp38-macosx_10_16_x86_64.whl" pip install absl-py astunparse flatbuffers gast google_pasta keras_preprocessing opt_einsum protobuf tensorflow_estimator termcolor typing_extensions wrapt wheel tensorboard typeguard pip install "/Users/username/Downloads/tensorflow_macos/x86_64/tensorflow_macos-0.1a0-cp38-cp38-macosx_10_16_x86_64.whl"

Source: #7 (comment)

To Install arm64 into virtualenv This would give the fastest speed since it uses the native arm64, but it doesn't support many libraries at the moment, including pandas.

To install:

  1. With tensorflow_macos unzipped in the Downloads folder
  2. In Terminal: browse to Downloads folder
  3. Run: /bin/bash ./tensorflow_macos/install_venv.sh --prompt
  4. Key in your desired path or just Enter for the default path

To Run:

  1. In a new terminal run: source /Users/username/tensorflow_macos_venv/bin/activate
  2. python3 "your_python_file.py"

To toggle between CPU and GPU runs adjust the device_name below: from tensorflow.python.compiler.mlcompute import mlcompute mlcompute.set_mlc_device(device_name='gpu')

To make it faster, turn off the default Eager mode with the following line: tf.compat.v1.disable_eager_execution()

Run Speed Comparison for MNIST character recognition: Anaconda Spyder (x86_64, EagerOff): 200-256us/sample Terminal (arm64, CPU, EagerOff): 22-63us/sample Terminal (arm64, GPU, EagerOff): 137-166us/sample

Tried so many online tutorials to install, finally found a one worked !

gabirelasanchezzz commented 3 years ago

4. pip install "/Users/username/Downloads/tensorflow_macos/x86_64/h5py-2.10.0-cp38-cp38-macosx_10_16_x86_64.whl"

Hi, sorry this does not work for me either. The next error pops up:

ERROR: grpcio-1.33.2-cp38-cp38-macosx_10_16_x86_64.whl is not a supported wheel on this platform.

mwidjaja1 commented 3 years ago

@gabirelasanchezzz You're using an Intel wheel file but this thread is for M1 computers.

gabirelasanchezzz commented 3 years ago

@mwidjaja1 I do have an M1. I just was following the instructions @denalist provided.

mwidjaja1 commented 3 years ago

Ahh gotcha. Yeah I'm not sure how to help you then. The route @denalist took, while legitimate for sure, does not use a native M1 Python. It uses Rosetta to build an Intel version of Python, for M1. Upside, installation should be easier and the entire world of pip + conda is at your fingertips. Downside, you won't get any of the m1 Processor + GPU efficiencies, and then situations like these. My guess is your Mac is confused between using the M1 version of Python it came with vs. the intel version you tried to download.

I did write my own tutorial for M1 Python + TensorFlow at #153 but I confess, it does get a little bit tricky because it is a strange scenario. In that path you would get the M1 optimizations but not every package will work, though most of the 'popular' packages like Numpy/Scipy would work.

denalist commented 3 years ago

@gabirelasanchezzz Please read it before making the conclusion, I didnt provide the solution. and the thread is clear enough says "M1".

gabirelasanchezzz commented 3 years ago

@mwidjaja1 I just went to your tutorial and started following the steps. When I eventually reached: "file $(which python)" in the terminal, Mach-O 64-bit executable x86_64 appears, but I do have an M1, so as you said, my Mac is confused. Do you know any way to fix this??

mwidjaja1 commented 3 years ago

Does which python direct you to the miniforge3 version of Python or some other path?

gabirelasanchezzz commented 3 years ago

it directs me to /opt/anaconda3/bin/python

mwidjaja1 commented 3 years ago

Please follow that section because it tells you the appropriate steps to redirect your Python to miniforge3. The guide has a lot of words but that's because I know there are a lot of ways things can go wrong. The paragraph you should have followed that'd fix that is:

If you did all that, set your environment paths to Miniforge's Python Installation. To do that, you need to figure out where conda was installed to (it's probably ~/miniforge3/condabin/conda) and then run ~/miniforge3/condabin/conda init in your terminal.

vivek9patel commented 3 years ago

Hey guys, For me, it was a python version error. I was using python=3.9.5 (latest) version, and the wheel file is supported in 3.8.* version as the file name (cp38) says. So I solved it using conda install python=3.8.5 in my conda environment. And it works fine!