search

tensorflow / probability

Probabilistic reasoning and statistical analysis in TensorFlow
https://www.tensorflow.org/probability/
Apache License 2.0
4.24k stars 1.09k forks source link

TF Probability for M1 Mac #1277

Open chiennifer opened 3 years ago

chiennifer commented 3 years ago

Are there are any pre-compiled whls available for M1 (arm) architecture?

davmre commented 3 years ago

TFP itself is pure Python so there's nothing in particular to build; it's possible it would 'just work' given a compatible TF build. It'd be great to hear if anyone has experience trying this.

Dave

On Sun, Mar 21, 2021 at 3:37 PM Jennifer Chien @.***> wrote:

Are there are any pre-compiled whls available for M1 (arm) architecture?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tensorflow/probability/issues/1277, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHSFCU6A543G6VGONOXQ43TEZYITANCNFSM4ZSA3DRQ .

ollie-bell commented 2 years ago

Are there any official instructions available for native installation of TFP for an M1 MacBook? I have followed the apple recommended installation steps for native installation of Tensorflow via Miniforge, but after that a pip install tensorflow-probability gives errors related to building the wheel for dm-tree.

Collecting tensorflow-probability
  Using cached tensorflow_probability-0.14.1-py2.py3-none-any.whl (5.7 MB)
Requirement already satisfied: six>=1.10.0 in /opt/homebrew/Caskroom/miniforge/base/envs/tensorflowM1/lib/python3.9/site-packages (from tensorflow-probability) (1.15.0)
Collecting dm-tree
  Using cached dm-tree-0.1.6.tar.gz (33 kB)
  Preparing metadata (setup.py) ... done
Collecting cloudpickle>=1.3
  Using cached cloudpickle-2.0.0-py3-none-any.whl (25 kB)
Requirement already satisfied: absl-py in /opt/homebrew/Caskroom/miniforge/base/envs/tensorflowM1/lib/python3.9/site-packages (from tensorflow-probability) (0.10.0)
Collecting decorator
  Downloading decorator-5.1.0-py3-none-any.whl (9.1 kB)
Requirement already satisfied: numpy>=1.13.3 in /opt/homebrew/Caskroom/miniforge/base/envs/tensorflowM1/lib/python3.9/site-packages (from tensorflow-probability) (1.19.5)
Requirement already satisfied: gast>=0.3.2 in /opt/homebrew/Caskroom/miniforge/base/envs/tensorflowM1/lib/python3.9/site-packages (from tensorflow-probability) (0.4.0)
Building wheels for collected packages: dm-tree
  Building wheel for dm-tree (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /opt/homebrew/Caskroom/miniforge/base/envs/tensorflowM1/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/3p/8d38mt7s4pgclp_dnvrdp5x40000gp/T/pip-install-hrv39luy/dm-tree_967ea78baa53432fac29a091dd157eb6/setup.py'"'"'; __file__='"'"'/private/var/folders/3p/8d38mt7s4pgclp_dnvrdp5x40000gp/T/pip-install-hrv39luy/dm-tree_967ea78baa53432fac29a091dd157eb6/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/3p/8d38mt7s4pgclp_dnvrdp5x40000gp/T/pip-wheel-3mc6hux2
       cwd: /private/var/folders/3p/8d38mt7s4pgclp_dnvrdp5x40000gp/T/pip-install-hrv39luy/dm-tree_967ea78baa53432fac29a091dd157eb6/
  Complete output (71 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-11.0-arm64-3.9
  creating build/lib.macosx-11.0-arm64-3.9/tree
  copying tree/__init__.py -> build/lib.macosx-11.0-arm64-3.9/tree
  copying tree/tree_test.py -> build/lib.macosx-11.0-arm64-3.9/tree
  copying tree/tree_benchmark.py -> build/lib.macosx-11.0-arm64-3.9/tree
  running build_ext
  bazel build //tree:_tree --symlink_prefix=build/temp.macosx-11.0-arm64-3.9/bazel- --compilation_mode=opt
  Starting local Bazel server and connecting to it...
  Loading:
  Loading: 0 packages loaded
  Analyzing: target //tree:_tree (1 packages loaded, 0 targets configured)
  Analyzing: target //tree:_tree (15 packages loaded, 35 targets configured)
  Analyzing: target //tree:_tree (18 packages loaded, 35 targets configured)
  Analyzing: target //tree:_tree (18 packages loaded, 35 targets configured)
  Analyzing: target //tree:_tree (18 packages loaded, 35 targets configured)
  INFO: Analyzed target //tree:_tree (25 packages loaded, 362 targets configured).
  INFO: Found 1 target...
  [0 / 31] [Prepa] Writing file external/com_google_absl/absl/strings/strings.cppmap
  [39 / 57] Compiling tree/tree.cc; 0s darwin-sandbox ... (8 actions, 7 running)
  [60 / 61] [Prepa] Linking tree/_tree.so
  Target //tree:_tree up-to-date (nothing to build)
  INFO: Elapsed time: 10.059s, Critical Path: 2.72s
  INFO: 61 processes: 27 internal, 34 darwin-sandbox.
  INFO: Build completed successfully, 61 total actions
  INFO: Build completed successfully, 61 total actions
  /opt/homebrew/Caskroom/miniforge/base/envs/tensorflowM1/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
    warnings.warn(
  installing to build/bdist.macosx-11.0-arm64/wheel
  running install
  running install_lib
  creating build/bdist.macosx-11.0-arm64
  creating build/bdist.macosx-11.0-arm64/wheel
  creating build/bdist.macosx-11.0-arm64/wheel/tree
  copying build/lib.macosx-11.0-arm64-3.9/tree/__init__.py -> build/bdist.macosx-11.0-arm64/wheel/tree
  copying build/lib.macosx-11.0-arm64-3.9/tree/tree_test.py -> build/bdist.macosx-11.0-arm64/wheel/tree
  copying build/lib.macosx-11.0-arm64-3.9/tree/tree_benchmark.py -> build/bdist.macosx-11.0-arm64/wheel/tree
  copying build/lib.macosx-11.0-arm64-3.9/tree/_tree.cpython-39-darwin.so -> build/bdist.macosx-11.0-arm64/wheel/tree
  running install_egg_info
  running egg_info
  writing dm_tree.egg-info/PKG-INFO
  writing dependency_links to dm_tree.egg-info/dependency_links.txt
  writing requirements to dm_tree.egg-info/requires.txt
  writing top-level names to dm_tree.egg-info/top_level.txt
  reading manifest file 'dm_tree.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  adding license file 'LICENSE'
  writing manifest file 'dm_tree.egg-info/SOURCES.txt'
  Copying dm_tree.egg-info to build/bdist.macosx-11.0-arm64/wheel/dm_tree-0.1.6-py3.9.egg-info
  running install_scripts
  [WARNING] This wheel needs a higher macOS version than the version your Python interpreter is compiled against.  To silence this warning, set MACOSX_DEPLOYMENT_TARGET to at least 11_3 or recreate these files with lower MACOSX_DEPLOYMENT_TARGET:
  build/bdist.macosx-11.0-arm64/wheel/tree/_tree.cpython-39-darwin.soTraceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/private/var/folders/3p/8d38mt7s4pgclp_dnvrdp5x40000gp/T/pip-install-hrv39luy/dm-tree_967ea78baa53432fac29a091dd157eb6/setup.py", line 118, in <module>
      setuptools.setup(
    File "/opt/homebrew/Caskroom/miniforge/base/envs/tensorflowM1/lib/python3.9/site-packages/setuptools/__init__.py", line 159, in setup
      return distutils.core.setup(**attrs)
    File "/opt/homebrew/Caskroom/miniforge/base/envs/tensorflowM1/lib/python3.9/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/opt/homebrew/Caskroom/miniforge/base/envs/tensorflowM1/lib/python3.9/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/opt/homebrew/Caskroom/miniforge/base/envs/tensorflowM1/lib/python3.9/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/opt/homebrew/Caskroom/miniforge/base/envs/tensorflowM1/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 328, in run
      impl_tag, abi_tag, plat_tag = self.get_tag()
    File "/opt/homebrew/Caskroom/miniforge/base/envs/tensorflowM1/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 278, in get_tag
      assert tag in supported_tags, "would build wheel with unsupported tag {}".format(tag)
  AssertionError: would build wheel with unsupported tag ('cp39', 'cp39', 'macosx_11_3_arm64')
  ----------------------------------------
  ERROR: Failed building wheel for dm-tree
  Running setup.py clean for dm-tree
Failed to build dm-tree
Installing collected packages: dm-tree, decorator, cloudpickle, tensorflow-probability
    Running setup.py install for dm-tree ... done
  DEPRECATION: dm-tree was installed using the legacy 'setup.py install' method, because a wheel could not be built for it. A possible replacement is to fix the wheel build issue reported above. Discussion can be found at https://github.com/pypa/pip/issues/8368
Successfully installed cloudpickle-2.0.0 decorator-5.1.0 dm-tree-0.1.6 tensorflow-probability-0.14.1

I can run the small introductory code snippet on https://www.tensorflow.org/probability, except I get a message after running features = tfp.distributions.Normal(loc=0., scale=1.).sample(int(100e3)). It seems that TensorFlow isn't picking up the GPU even with tensorflow-metal installed?

>>> features = tfp.distributions.Normal(loc=0., scale=1.).sample(int(100e3))
Metal device set to: Apple M1

systemMemory: 8.00 GB
maxCacheSize: 2.67 GB

2021-11-14 20:09:25.839625: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2021-11-14 20:09:25.840817: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)