search

tensorflow / io

Dataset, streaming, and file system extensions maintained by TensorFlow SIG-IO
Apache License 2.0
701 stars 281 forks source link

Building tensorflow-io locally on Mac M2 #1802

Open AndrewKovalenko opened 1 year ago

AndrewKovalenko commented 1 year ago

I try to build tensorflow-io locally on my Mac with Silicon M2 by running 

$(which python) setup.py -q bdist_wheel
$(which python) setup.py -q bdist_wheel --project tensorflow_io_gcs_filesystem

which gives me 

Project: tensorflow-io
Exclude: ['tests', 'tests.*', 'tensorflow_io_gcs_filesystem', 'tensorflow_io_gcs_filesystem.*']
Install Requires: ['tensorflow-io-gcs-filesystem==0.32.0']
Project Rootpath: tensorflow_io
/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer, pypa/build or
        other standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()

Project: tensorflow-io-gcs-filesystem
Exclude: ['tests', 'tests.*', 'tensorflow_io', 'tensorflow_io.*']
Install Requires: []
Project Rootpath: tensorflow_io_gcs_filesystem
/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer, pypa/build or
        other standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()

and generates two files in dist so I assume that build was successful. 

-rw-r--r--   1 dev  staff  224569 May 22 12:19 tensorflow_io-0.32.0-cp310-cp310-macosx_11_0_arm64.whl
-rw-r--r--   1 dev  staff  224735 May 22 12:21 tensorflow_io_gcs_filesystem-0.32.0-cp310-cp310-macosx_11_0_arm64.whl

then I install both packages with 

$(which python) -m pip install --no-deps tensorflow_io-0.32.0-cp310-cp310-macosx_11_0_arm64.whl --force-reinstall
$(which python) -m pip install --no-deps tensorflow_io_gcs_filesystem-0.32.0-cp310-cp310-macosx_11_0_arm64.whl --force-reinstall

and both commands give successful result but when I try to import tensorflow_io I get the following error:

[/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:98](https://file+.vscode-resource.vscode-cdn.net/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:98): UserWarning: unable to load libtensorflow_io_plugins.so: unable to open file: libtensorflow_io_plugins.so, from paths: ['[/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so](https://file+.vscode-resource.vscode-cdn.net/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so)']
caused by: ["[Errno 2] The file to load file system plugin from does not exist.: '[/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so](https://file+.vscode-resource.vscode-cdn.net/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so)'"]
  warnings.warn(f"unable to load libtensorflow_io_plugins.so: {e}")
[/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:104](https://file+.vscode-resource.vscode-cdn.net/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:104): UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so, from paths: ['[/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so](https://file+.vscode-resource.vscode-cdn.net/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so)']
caused by: ["dlopen([/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so](https://file+.vscode-resource.vscode-cdn.net/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so), 0x0006): tried: '[/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so](https://file+.vscode-resource.vscode-cdn.net/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so)' (no such file), '[/System/Volumes/Preboot/Cryptexes/OS/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so](https://file+.vscode-resource.vscode-cdn.net/System/Volumes/Preboot/Cryptexes/OS/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so)' (no such file), '[/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so](https://file+.vscode-resource.vscode-cdn.net/Users/dev/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so)' (no such file)"]
  warnings.warn(f"file system plugins are not loaded: {e}")

and attempt to use it fails due to libtensorflow_io.so file missing:

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[2], line 1
----> 1 audio = tfio.audio.AudioIOTensor('gs:[//cloud-samples-tests/speech/brooklyn.flac](https://cloud-samples-tests/speech/brooklyn.flac)')
      2 print(audio)

File [~/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/audio_ops.py:671](https://file+.vscode-resource.vscode-cdn.net/Users/dev/development/repositories/copilot/researches/~/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/audio_ops.py:671), in AudioIOTensor.__init__(self, filename, dtype)
    669 if not tf.executing_eagerly():
    670     assert dtype is not None, "dtype must be provided in graph mode"
--> 671 resource = core_ops.io_audio_readable_init(filename)
    672 if tf.executing_eagerly():
    673     shape, dtype, rate = core_ops.io_audio_readable_spec(resource)

File [~/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:88](https://file+.vscode-resource.vscode-cdn.net/Users/dev/development/repositories/copilot/researches/~/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:88), in LazyLoader.__getattr__(self, attrb)
     87 def __getattr__(self, attrb):
---> 88     return getattr(self._load(), attrb)

File [~/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:84](https://file+.vscode-resource.vscode-cdn.net/Users/dev/development/repositories/copilot/researches/~/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:84), in LazyLoader._load(self)
     82 def _load(self):
     83     if self._mod is None:
---> 84         self._mod = _load_library(self._library)
     85     return self._mod

File [~/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:69](https://file+.vscode-resource.vscode-cdn.net/Users/dev/development/repositories/copilot/researches/~/miniconda3/envs/tensor-flow-2-12/lib/python3.10/site-packages/tensorflow_io/python/ops/__init__.py:69), in _load_library(filename, lib)
     67     except (tf.errors.NotFoundError, OSError) as e:
...
     71     + f"{filename}, from paths: {filenames}\ncaused by: {errs}"
     72 )

Versions:

ProductName:        macOS
ProductVersion:     13.3
BuildVersion:       22E252

Python 3.10.11
pip 23.1.2
trueToastedCode commented 1 year ago

rnings.warn(f"file system plugins are not loaded: {e}")

Replica of #1755

Hopefully it will get fixed soon. It's a MacOS issue... works on Linux/arm64

alexander-beedie commented 1 year ago

Unfortunately Apple Silicon support has been an issue here for a long time; totally understandable when M1 was first released, but it's not going to be much longer until we have M3 😢 (See: https://github.com/tensorflow/io/issues/1625 from Feb 2022).