Is libtensorflow_framework.so required in this mode of custom op?

[firejq]

Tensorflow: 1.13.1

In tensorflow's .bazelrc file, there is a config setting framework_shared_object, which is default true so that we can get a libtensorflow_framework.so in the wheel package after compiling.

Howerver, because of certerns reasons, I built my tensorflow wheel package while framework_shared_object is false and there is no libtensorflow_framework.so in the wheel package, which caused that I cannot built my custom op according to this repo.

In a word, Is libtensorflow_framework.so required in this mode of custom op? Is there any other ways to compiling the custom op without this so file?

@yifeif @vatai Could you please give some suggestions? Thanks a lot!

[yifeif]

As long as pywrap_tensorflow expose all the symbols (cc: @gunan) for the custom ops you need, you should be able to do the same linking against pywrap_tensorflow. As a matter of fact, we are currently doing that for Windows as Windows has framework_shared_object off.

[firejq]

@yifeif Thanks for your quick reply! But I still have some questions about this:

1. Do you mean that when framework_shared_object is off all the symbol implements are in the _pywrap_tensorflow_internal.so, and when framework_shared_object is on all the symbol implements are in libtensorflow_framework.so?

2. Why is framework_shared_object default off only on Windows? Why not other platforms?

I am sorry that my question may be a bit confusing since I still don’t understand clearly..

[gunan]

1 - Yes, that is correct. If we set framework_shared_object to false, all symbols are a part of _pywrap_tensorflow_internal. When framework_shared_object is set to true, only the symbols that are needed for custom ops and kernels are in libtensorflow_framework and the rest are still in _pywrap_tensorflow_internal.

2 - On windows, when we turn it on we run into issues due to windows' limit on how many symbols can be exported from a shared object. We are working to have a comprehensive symbol filter, and when we do that we will enable framework_shared_object on windows as well.

[firejq]

@gunan Thanks for your reply!

It's understandable to me but there is probably a slip of a pen in A-1:

and the rest are still in libtensorflow_framework.

-> and the rest are still in _pywrap_tensorflow_internal.

[gunan]

Thank you very much! I have corrected my comment to avoid confusion later.