Closed jnghhk closed 6 years ago
Hmm.. Looks like an ABI mismatch problem. Possibly your tensorflow wheel was built with a different gcc version from what you are using to build sbnet custom ops.
https://github.com/tensorflow/tensorflow/issues/9137 In this TF issue they are missing the same exact symbol and the solution appears to be using -D_GLIBCXX_USE_CXX11_ABI=0 during build for both the TF pip and the custom op.
TensorFlow installed from (source or binary): pip (conda env) TensorFlow version (use command below): 1.8.0 Python version: 2.7 GCC/Compiler version (if compiling from source):5.4.0 CUDA/cuDNN version:9/7.1.2 GPU model and memory: GTX1060 6G Exact command to reproduce: make test
cd ../benchmark && bash run_all_unittests.bash # unit tests sparse_res_block_tests (unittest.loader.ModuleImportFailure) ... ERROR reduce_mask_tests (unittest.loader.ModuleImportFailure) ... ERROR sparse_conv_tests (unittest.loader.ModuleImportFailure) ... ERROR sparse_scatter_tests (unittest.loader.ModuleImportFailure) ... ERROR sparse_gather_tests (unittest.loader.ModuleImportFailure) ... ERROR test_calc_out_size (tf_conv_dims_tests.CalcOutSizeDeconvTests) ... ok test_session (tf_conv_dims_tests.CalcOutSizeDeconvTests) Returns a TensorFlow Session for use in executing tests. ... ok test_calc_out_size (tf_conv_dims_tests.CalcOutSizeTests) ... ok test_session (tf_conv_dims_tests.CalcOutSizeTests) Returns a TensorFlow Session for use in executing tests. ... ok test_calc_padding (tf_conv_dims_tests.CalcPaddingTests) ... ok test_calc_padding_err_ksize_list (tf_conv_dims_tests.CalcPaddingTests) ... ok test_calc_padding_err_strides_list (tf_conv_dims_tests.CalcPaddingTests) ... ok test_calc_padding_err_strides_tensor (tf_conv_dims_tests.CalcPaddingTests) ... ok test_calc_padding_stride (tf_conv_dims_tests.CalcPaddingTests) ... ok test_calc_padding_valid (tf_conv_dims_tests.CalcPaddingTests) ... ok test_session (tf_conv_dims_tests.CalcPaddingTests) Returns a TensorFlow Session for use in executing tests. ... ok
====================================================================== ERROR: sparse_res_block_tests (unittest.loader.ModuleImportFailure)
ImportError: Failed to import test module: sparse_res_block_tests Traceback (most recent call last): File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 254, in _find_tests module = self._get_module_from_name(name) File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name import(name) File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_res_block_tests.py", line 32, in
from sparse_conv_lib import _get_offset_array
File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_conv_lib.py", line 74, in
sbnet_module = tf.load_op_library('../sbnet_ops/libsbnet.so')
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
NotFoundError: ../sbnet_ops/libsbnet.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv
====================================================================== ERROR: reduce_mask_tests (unittest.loader.ModuleImportFailure)
ImportError: Failed to import test module: reduce_mask_tests Traceback (most recent call last): File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 254, in _find_tests module = self._get_module_from_name(name) File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name import(name) File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/reduce_mask_tests.py", line 25, in
from sparse_conv_lib import convert_mask_to_indices, convert_mask_to_indices_custom
File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_conv_lib.py", line 74, in
sbnet_module = tf.load_op_library('../sbnet_ops/libsbnet.so')
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
NotFoundError: ../sbnet_ops/libsbnet.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv
====================================================================== ERROR: sparse_conv_tests (unittest.loader.ModuleImportFailure)
ImportError: Failed to import test module: sparse_conv_tests Traceback (most recent call last): File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 254, in _find_tests module = self._get_module_from_name(name) File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name import(name) File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_conv_tests.py", line 29, in
from sparse_conv_lib import _get_offset_array
File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_conv_lib.py", line 74, in
sbnet_module = tf.load_op_library('../sbnet_ops/libsbnet.so')
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
NotFoundError: ../sbnet_ops/libsbnet.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv
====================================================================== ERROR: sparse_scatter_tests (unittest.loader.ModuleImportFailure)
ImportError: Failed to import test module: sparse_scatter_tests Traceback (most recent call last): File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 254, in _find_tests module = self._get_module_from_name(name) File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name import(name) File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_scatter_tests.py", line 26, in
from sparse_conv_lib import sbnet_module
File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_conv_lib.py", line 74, in
sbnet_module = tf.load_op_library('../sbnet_ops/libsbnet.so')
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
NotFoundError: ../sbnet_ops/libsbnet.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv
====================================================================== ERROR: sparse_gather_tests (unittest.loader.ModuleImportFailure)
ImportError: Failed to import test module: sparse_gather_tests Traceback (most recent call last): File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 254, in _find_tests module = self._get_module_from_name(name) File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name import(name) File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_gather_tests.py", line 26, in
from sparse_conv_lib import convert_mask_to_block_indices, convert_mask_to_indices_custom
File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_conv_lib.py", line 74, in
sbnet_module = tf.load_op_library('../sbnet_ops/libsbnet.so')
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
NotFoundError: ../sbnet_ops/libsbnet.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv
Ran 16 tests in 4.775s
FAILED (errors=5) Makefile:14: recipe for target 'test' failed make: *** [test] Error 1