python venv with TF & ONNX

[seanshpark]

Inside homebrew.

[seanshpark]

pip3 install --user virtualenv

or

pip3 install virtualenv

with homebrew?

pip3 install virtualenv

seems to install OK.

Prerequisites

xcode-select --install

brew install cmake
brew install protobuf protobuf-c

[seanshpark]

cd ~/hdd/py3venv

virtualenv -p python3.8 one_tf3

source one_tf3/bin/activate

or

source ~/hdd/py3venv/one_tf3/bin/activate

packages

pip3 install pytest

[seanshpark]

install packages from one-prepare-venv WIP

xcode-select --install

pip3 install --upgrade pip setuptools

CPATH=/Volumes/SSD/homebrew/include \
LIBRARY_PATH=/Volumes/SSD/homebrew/lib \
pip3 install tensorflow-macos==2.7.0

pip3 install Pillow==6.2.2
--> X

pip3 install torch==1.10.1 -f https://download.pytorch.org/whl/torch_stable.html

pip3 install keras==2.6.0
pip3 install onnx==1.10.1

# install tensorflow_addons from source (see below)
pip3 install artifacts/tensorflow_addons-*.whl

pip3 install onnx-tf==1.9.0
# pip3 install onnxruntime==1.8.1
# install onnxruntime from source (see below)
pip3 install build/MacOS/RelWithDebInfo/dist/onnxruntime-1.11.0-cp38-cp38-macosx_12_0_arm64.whl

there is no tensorflow-cpu for M1 😭

pillow ?

Collecting Pillow==6.2.2
  Downloading Pillow-6.2.2.tar.gz (37.8 MB)
     |████████████████████████████████| 37.8 MB 12.3 MB/s            
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: Pillow
  Building wheel for Pillow (setup.py) ... error

There is -macos

pip3 install tensorflow-macos==2.6.0

clang: error: the clang compiler does not support 'faltivec', please use -maltivec and include altivec.h explicitly

maybe use python3.8 ?

[seanshpark]

brew install python@3.8
brew unlink python
brew link python@3.8

pip3 install tensorflow-macos==2.6.0

--> X

pip3 install tensorflow-macos==2.7.0

Loading library to get build settings and version: libhdf5.dylib error: Unable to load dependency HDF5, make sure HDF5 is installed properly

brew install hdf5

pip3 install tensorflow-macos==2.7.0

./h5py/api_compat.h:27:10: fatal error: 'hdf5.h' file not found

CPATH=/Volumes/SSD/homebrew/include \
pip3 install tensorflow-macos==2.7.0

ld: library not found for -lhdf5

CPATH=/Volumes/SSD/homebrew/include \
LIBRARY_PATH=/Volumes/SSD/homebrew/lib \
pip3 install tensorflow-macos==2.7.0

-> OK

Successfully installed absl-py-1.0.0 astunparse-1.6.3 cachetools-4.2.4 certifi-2021.10.8 charset-normalizer-2.0.10 flatbuffers-2.0 gast-0.4.0 google-auth-2.3.3 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 grpcio-1.43.0 h5py-3.6.0 idna-3.3 importlib-metadata-4.10.1 keras-2.7.0 keras-preprocessing-1.1.2 libclang-12.0.0 markdown-3.3.6 numpy-1.22.1 oauthlib-3.1.1 opt-einsum-3.3.0 protobuf-3.19.3 pyasn1-0.4.8 pyasn1-modules-0.2.8 requests-2.27.1 requests-oauthlib-1.3.0 rsa-4.8 tensorboard-2.7.0 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 tensorflow-estimator-2.7.0 tensorflow-macos-2.7.0 termcolor-1.1.0 typing-extensions-4.0.1 urllib3-1.26.8 werkzeug-2.0.2 wrapt-1.13.3 zipp-3.7.0

[seanshpark]

CPATH=/Volumes/SSD/homebrew/include \
LIBRARY_PATH=/Volumes/SSD/homebrew/lib \
pip3 install Pillow==6.2.2

The headers or library files could not be found for zlib, a required dependency when compiling Pillow from source.

Not required... skip this.

[seanshpark]

pip3 install onnx==1.10.1

--> requires cmake

Protobuf compiler not found

brew install cmake
brew install protobuf protobuf-c

[seanshpark]

pip3 install onnx-tf==1.9.0

ERROR: Could not find a version that satisfies the requirement tensorflow-addons (from onnx-tf) (from versions: none) ERROR: No matching distribution found for tensorflow-addons

--> https://pypi.org/project/tensorflow-addons/#files

there is no 3.8 arm64

--> https://www.tensorflow.org/addons/overview#installing_from_source

need bazel: #12

python3 ./configure.py

bazel build build_pip_pkg
bazel-bin/build_pip_pkg artifacts

pip install artifacts/tensorflow_addons-*.whl

[seanshpark]

pip3 install onnxruntime==1.8.1

ERROR: Could not find a version that satisfies the requirement onnxruntime==1.8.1 (from versions: none) ERROR: No matching distribution found for onnxruntime==1.8.1

onnxruntime build from source

• https://onnxruntime.ai/docs/build/inferencing#cpu

git clone --recursive https://github.com/Microsoft/onnxruntime
cd onnxruntime

./build.sh --config RelWithDebInfo --build_shared_lib --osx_arch arm64

./build.sh --config RelWithDebInfo --build_shared_lib --osx_arch arm64 --build_wheel

--> OK

pip3 install build/MacOS/RelWithDebInfo/dist/onnxruntime-1.11.0-cp38-cp38-macosx_12_0_arm64.whl