tensorflow / custom-op

Guide for building custom op for TensorFlow
Apache License 2.0
378 stars 115 forks source link

Need a working example how to build the custom ops for new tensorflow versions - like 2.8. #111

Open ml-scripts opened 2 years ago

ml-scripts commented 2 years ago

Need a working example how to build the custom ops for new tensorflow versions - like 2.8.

Current docker container come with old python version (3.6) and won't even allow me to pip install tensorflow-cpu==2.8.2.

There is a need to either update the docker containers for building custom ops or provide updated documentation.

Also the instructions run out of sync with https://www.tensorflow.org/guide/create_op which now uses tf_custom_op_library

Thank you

alkhwarizmi commented 1 year ago

Well, I also was looking for updated examples targeting tf 2.13, python 3.11 but it seems there is not much out thre. This project could be the best starting point. Or there are other resources or manuals?

ml-scripts commented 1 year ago

I managed to workaround and use newer python version (3.7) and tensorflow 2.8 and 2.9. It wasn't pretty but it worked. Unsure about all the things I needed to do but here is what I remember:

Inside of docker container I needed to run:

sudo update-alternatives --install /usr/bin/python python /usr/local/bin/python3.7 1
python -m pip install --upgrade pip
python -m pip install --upgrade tensorflow-cpu=="$TF_VERSION"
echo "Y Y Y" | ./configure.sh

That changed my python version and allowed to install the tf version I needed.

I also needed to make some changes to project build files: my .bazelrc

build:cuda --define=using_cuda=true --define=using_cuda_nvcc=true
build:manylinux2010cuda100 --crosstool_top=//third_party/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010-nvcc-cuda10.0:toolchain
build:manylinux2010cuda101 --crosstool_top=//third_party/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010-nvcc-cuda10.1:toolchain
build --spawn_strategy=standalone
build --strategy=Genrule=standalone
build -c opt
build --action_env TF_HEADER_DIR="/usr/local/lib/python3.7/site-packages/tensorflow/include"
build --action_env TF_SHARED_LIBRARY_DIR="/usr/local/lib/python3.7/site-packages/tensorflow"
build --action_env TF_SHARED_LIBRARY_NAME="libtensorflow_framework.so.2"
build --action_env TF_NEED_CUDA="0"
build --config=manylinux2010cuda101
test --config=manylinux2010cuda101

in configure.sh I needed to change PIP="pip3.7" and I needed to change -std=c++11 to -std=c++14 and "libtensorflow_framework.so" to "libtensorflow_framework.so.2" in bunch of places.

Sadly I don't have better instructions or time to create easy to follow and reproduce instructions. Good luck.