[nGraph] mkldnn::relu_forward and mkldnn::relu_backward error: expected type-specifier

[byronyi]

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Debian 9.4
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy): N/A
• TensorFlow installed from (source or binary): source
• TensorFlow version: dc77be8f92fd7b7ac57d69e35c4d0dbd994bf1a0
• Python version: 3.5.3
• Installed using virtualenv? pip? conda?: virtualenv
• Bazel version (if compiling from source): 0.24.1
• GCC/Compiler version (if compiling from source): gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
• CPU model: Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz

Describe the problem

bazel build --config=opt --config=mkl --config=numa --config=ngraph //tensorflow/tools/pip_package:build_pip_package

Any other info / logs

ERROR: /root/.cache/bazel/_bazel_root/5f3660b20527c5657e75cad53b138917/external/ngraph/BUILD.bazel:11:1: C++ compilation of rule '@ngraph//:ngraph_cpu_backend' failed (Exit 1) 
external/ngraph/src/ngraph/runtime/cpu/mkldnn_emitter.cpp: In member function 'size_t ngraph::runtime::cpu::MKLDNNEmitter::build_relu_forward(const mkldnn::memory::desc&, const mkldnn::memory::desc&)': 
external/ngraph/src/ngraph/runtime/cpu/mkldnn_emitter.cpp:810:51: error: expected type-specifier
      size_t primitive_index = insert_primitive(new mkldnn::relu_forward(
                                                    ^~~~~~
external/ngraph/src/ngraph/runtime/cpu/mkldnn_emitter.cpp: In member function 'size_t ngraph::runtime::cpu::MKLDNNEmitter::build_relu_backward(const mkldnn::memory::desc&, const mkldnn::memory::desc&, const mkldnn::memory::desc&)':
external/ngraph/src/ngraph/runtime/cpu/mkldnn_emitter.cpp:828:51: error: expected type-specifier
      size_t primitive_index = insert_primitive(new mkldnn::relu_backward(
                                                    ^~~~~~
Target //tensorflow/tools/pip_package:build_pip_package failed to build 
Use --verbose_failures to see the command lines of failed build steps. 
INFO: Elapsed time: 95.354s, Critical Path: 46.01s 
INFO: 1419 processes: 1419 local. 
FAILED: Build did NOT complete successfully

r1.14 branch:

ERROR: /root/.cache/bazel/_bazel_root/5f3660b20527c5657e75cad53b138917/external/ngraph/BUILD.bazel:11:1: C++ compilation of rule '@ngraph//:ngraph_cpu_backend' failed (Exit 1)
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp: In function 'std::map<mkldnn::memory::format, const std::__cxx11::basic_string<char> >& ngraph::runtime::cpu::mkldnn_utils::get_mkldnn_format_string_map()':
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:162:10: error: 'ldigo_p' is not a member of 'mkldnn::memory::format'
         {memory::format::ldigo_p, "memory::format::ldigo_p"},
          ^~~~~~
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:164:10: error: 'ldgoi_p' is not a member of 'mkldnn::memory::format'
         {memory::format::ldgoi_p, "memory::format::ldgoi_p"},
          ^~~~~~
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:168:5: error: no matching function for call to 'std::map<mkldnn::memory::format, const std::__cxx11::basic_string<char> >::map(<brace-enclosed initializer list>)'
     };
     ^
In file included from /usr/include/c++/6/map:61:0,
                 from external/ngraph/src/ngraph/autodiff/adjoints.hpp:19,
                 from external/ngraph/src/ngraph/node.hpp:32,
                 from external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:22:
/usr/include/c++/6/bits/stl_map.h:273:9: note: candidate: template<class _InputIterator> std::map<_Key, _Tp, _Compare, _Alloc>::map(_InputIterator, _InputIterator, const _Compare&, const allocator_type&)
         map(_InputIterator __first, _InputIterator __last,
         ^~~
/usr/include/c++/6/bits/stl_map.h:273:9: note:   template argument deduction/substitution failed:
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:168:5: note:   candidate expects 4 arguments, 82 provided
     };
     ^
In file included from /usr/include/c++/6/map:61:0,
                 from external/ngraph/src/ngraph/autodiff/adjoints.hpp:19,
                 from external/ngraph/src/ngraph/node.hpp:32,
                 from external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:22:
/usr/include/c++/6/bits/stl_map.h:256:9: note: candidate: template<class _InputIterator> std::map<_Key, _Tp, _Compare, _Alloc>::map(_InputIterator, _InputIterator)
         map(_InputIterator __first, _InputIterator __last)
         ^~~
/usr/include/c++/6/bits/stl_map.h:256:9: note:   template argument deduction/substitution failed:
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:168:5: note:   candidate expects 2 arguments, 82 provided
     };
     ^
In file included from /usr/include/c++/6/map:61:0,
                 from external/ngraph/src/ngraph/autodiff/adjoints.hpp:19,
                 from external/ngraph/src/ngraph/node.hpp:32,
                 from external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:22:
/usr/include/c++/6/bits/stl_map.h:239:9: note: candidate: template<class _InputIterator> std::map<_Key, _Tp, _Compare, _Alloc>::map(_InputIterator, _InputIterator, const allocator_type&)
         map(_InputIterator __first, _InputIterator __last,
         ^~~
/usr/include/c++/6/bits/stl_map.h:239:9: note:   template argument deduction/substitution failed:
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:168:5: note:   candidate expects 3 arguments, 82 provided
     };
     ^
In file included from /usr/include/c++/6/map:61:0,
                 from external/ngraph/src/ngraph/autodiff/adjoints.hpp:19,
                 from external/ngraph/src/ngraph/node.hpp:32,
                 from external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:22:
/usr/include/c++/6/bits/stl_map.h:233:7: note: candidate: std::map<_Key, _Tp, _Compare, _Alloc>::map(std::initializer_list<std::pair<const _Key, _Tp> >, const allocator_type&) [with _Key = mkldnn::memory::format; _Tp = const std::__cxx11::basic_string<char>; _Compare = std::less<mkldnn::memory::format>; _Alloc = std::allocator<std::pair<const mkldnn::memory::format, const std::__cxx11::basic_string<char> > >; std::map<_Key, _Tp, _Compare, _Alloc>::allocator_type = std::allocator<std::pair<const mkldnn::memory::format, const std::__cxx11::basic_string<char> > >]
       map(initializer_list<value_type> __l, const allocator_type& __a)
       ^~~
/usr/include/c++/6/bits/stl_map.h:233:7: note:   candidate expects 2 arguments, 82 provided
/usr/include/c++/6/bits/stl_map.h:227:7: note: candidate: std::map<_Key, _Tp, _Compare, _Alloc>::map(std::map<_Key, _Tp, _Compare, _Alloc>&&, const allocator_type&) [with _Key = mkldnn::memory::format; _Tp = const std::__cxx11::basic_string<char>; _Compare = std::less<mkldnn::memory::format>; _Alloc = std::allocator<std::pair<const mkldnn::memory::format, const std::__cxx11::basic_string<char> > >; std::map<_Key, _Tp, _Compare, _Alloc>::allocator_type = std::allocator<std::pair<const mkldnn::memory::format, const std::__cxx11::basic_string<char> > >]
       map(map&& __m, const allocator_type& __a)
       ^~~
/usr/include/c++/6/bits/stl_map.h:227:7: note:   candidate expects 2 arguments, 82 provided
/usr/include/c++/6/bits/stl_map.h:223:7: note: candidate: std::map<_Key, _Tp, _Compare, _Alloc>::map(const std::map<_Key, _Tp, _Compare, _Alloc>&, const allocator_type&) [with _Key = mkldnn::memory::format; _Tp = const std::__cxx11::basic_string<char>; _Compare = std::less<mkldnn::memory::format>; _Alloc = std::allocator<std::pair<const mkldnn::memory::format, const std::__cxx11::basic_string<char> > >; std::map<_Key, _Tp, _Compare, _Alloc>::allocator_type = std::allocator<std::pair<const mkldnn::memory::format, const std::__cxx11::basic_string<char> > >]
       map(const map& __m, const allocator_type& __a)
       ^~~
/usr/include/c++/6/bits/stl_map.h:223:7: note:   candidate expects 2 arguments, 82 provided
/usr/include/c++/6/bits/stl_map.h:219:7: note: candidate: std::map<_Key, _Tp, _Compare, _Alloc>::map(const allocator_type&) [with _Key = mkldnn::memory::format; _Tp = const std::__cxx11::basic_string<char>; _Compare = std::less<mkldnn::memory::format>; _Alloc = std::allocator<std::pair<const mkldnn::memory::format, const std::__cxx11::basic_string<char> > >; std::map<_Key, _Tp, _Compare, _Alloc>::allocator_type = std::allocator<std::pair<const mkldnn::memory::format, const std::__cxx11::basic_string<char> > >]
       map(const allocator_type& __a)
       ^~~
/usr/include/c++/6/bits/stl_map.h:219:7: note:   candidate expects 1 argument, 82 provided
/usr/include/c++/6/bits/stl_map.h:211:7: note: candidate: std::map<_Key, _Tp, _Compare, _Alloc>::map(std::initializer_list<std::pair<const _Key, _Tp> >, const _Compare&, const allocator_type&) [with _Key = mkldnn::memory::format; _Tp = const std::__cxx11::basic_string<char>; _Compare = std::less<mkldnn::memory::format>; _Alloc = std::allocator<std::pair<const mkldnn::memory::format, const std::__cxx11::basic_string<char> > >; std::map<_Key, _Tp, _Compare, _Alloc>::allocator_type = std::allocator<std::pair<const mkldnn::memory::format, const std::__cxx11::basic_string<char> > >]
       map(initializer_list<value_type> __l,
       ^~~
/usr/include/c++/6/bits/stl_map.h:211:7: note:   candidate expects 3 arguments, 82 provided
/usr/include/c++/6/bits/stl_map.h:196:7: note: candidate: std::map<_Key, _Tp, _Compare, _Alloc>::map(std::map<_Key, _Tp, _Compare, _Alloc>&&) [with _Key = mkldnn::memory::format; _Tp = const std::__cxx11::basic_string<char>; _Compare = std::less<mkldnn::memory::format>; _Alloc = std::allocator<std::pair<const mkldnn::memory::format, const std::__cxx11::basic_string<char> > >]
       map(map&& __x)
       ^~~
/usr/include/c++/6/bits/stl_map.h:196:7: note:   candidate expects 1 argument, 82 provided
/usr/include/c++/6/bits/stl_map.h:185:7: note: candidate: std::map<_Key, _Tp, _Compare, _Alloc>::map(const std::map<_Key, _Tp, _Compare, _Alloc>&) [with _Key = mkldnn::memory::format; _Tp = const std::__cxx11::basic_string<char>; _Compare = std::less<mkldnn::memory::format>; _Alloc = std::allocator<std::pair<const mkldnn::memory::format, const std::__cxx11::basic_string<char> > >]
       map(const map& __x)
       ^~~
/usr/include/c++/6/bits/stl_map.h:185:7: note:   candidate expects 1 argument, 82 provided
/usr/include/c++/6/bits/stl_map.h:174:7: note: candidate: std::map<_Key, _Tp, _Compare, _Alloc>::map(const _Compare&, const allocator_type&) [with _Key = mkldnn::memory::format; _Tp = const std::__cxx11::basic_string<char>; _Compare = std::less<mkldnn::memory::format>; _Alloc = std::allocator<std::pair<const mkldnn::memory::format, const std::__cxx11::basic_string<char> > >; std::map<_Key, _Tp, _Compare, _Alloc>::allocator_type = std::allocator<std::pair<const mkldnn::memory::format, const std::__cxx11::basic_string<char> > >]
       map(const _Compare& __comp,
       ^~~
/usr/include/c++/6/bits/stl_map.h:174:7: note:   candidate expects 2 arguments, 82 provided
/usr/include/c++/6/bits/stl_map.h:162:7: note: candidate: std::map<_Key, _Tp, _Compare, _Alloc>::map() [with _Key = mkldnn::memory::format; _Tp = const std::__cxx11::basic_string<char>; _Compare = std::less<mkldnn::memory::format>; _Alloc = std::allocator<std::pair<const mkldnn::memory::format, const std::__cxx11::basic_string<char> > >]
       map()
       ^~~
/usr/include/c++/6/bits/stl_map.h:162:7: note:   candidate expects 0 arguments, 82 provided
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp: In function 'mkldnn::memory::desc ngraph::runtime::cpu::mkldnn_utils::rotate_blocked_md(const mkldnn::memory::desc&, const ngraph::AxisVector&)':
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:469:26: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     for (size_t i = 0; i < in.data.ndims; i++)
                        ~~^~~~~~~~~~~~~~~
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp: In function 'mkldnn::memory::desc ngraph::runtime::cpu::mkldnn_utils::squeeze_blocked_md(const mkldnn::memory::desc&, ngraph::AxisVector&)':
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:492:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     if (in.data.ndims <= axis_list.size())
         ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:513:33: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     for (size_t i = 0, j = 0; i < in.data.ndims; i++)
                               ~~^~~~~~~~~~~~~~~
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp: In function 'mkldnn::memory::desc ngraph::runtime::cpu::mkldnn_utils::expand_blocked_md(const mkldnn::memory::desc&, ngraph::AxisVector&)':
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:546:33: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     for (size_t i = 0, j = 0; j < md.ndims; j++)
                               ~~^~~~~~~~~~
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:567:42: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
                 for (size_t idx = 0; idx < in.data.ndims; idx++)
                                      ~~~~^~~~~~~~~~~~~~~
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp: In function 'bool ngraph::runtime::cpu::mkldnn_utils::is_mkldnn_padded_layout(const mkldnn::memory::desc&, const ngraph::AxisVector&)':
external/ngraph/src/ngraph/runtime/cpu/mkldnn_utils.cpp:630:26: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     for (size_t i = 0; i < in.data.ndims; i++)
                        ~~^~~~~~~~~~~~~~~
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 108.907s, Critical Path: 44.36s
INFO: 3128 processes: 3128 local.
FAILED: Build did NOT complete successfully

[byronyi]

Gentle ping @avijit-nervana

[byronyi]

Ping @agramesh1 as well; looks like ldigo_p got removed when upgrading MKLDNN to 0.18.

[avijit-nervana]

@byronyi There are two ways to use nGraph with TensorFlow - one is to compile nGraph with TensorFlow build - that you are attempting and the other is to build nGraph only and dynamically loading when running TensorFlow. The second approach is much easier to build and use. See the Build Option 2 here: https://github.com/tensorflow/ngraph-bridge#option-2-build-ngraph-bridge-with-binary-tensorflow-installation

Please try that and let us know if you run into any issues.

[byronyi]

@byronyi There are two ways to use nGraph with TensorFlow - one is to compile nGraph with TensorFlow build - that you are attempting and the other is to build nGraph only and dynamically loading when running TensorFlow. The second approach is much easier to build and use. See the Build Option 2 here: https://github.com/tensorflow/ngraph-bridge#option-2-build-ngraph-bridge-with-binary-tensorflow-installation

Please try that and let us know if you run into any issues.

Does that mean the Bazel option with the --opt=ngraph no longer be supported? It might better be removed entirely if so.

We build nGraph with TF as we did customize our internal TF build. The binary approach isn't always the best choice as TF does not provide a stable ABI now (it's moving to that goal as in the modular TF RFC but it's gonna take some time to be implemented). See https://github.com/tensorflow/tensorflow/issues/27067 for a potential problem.

[NeoZhangJianyu]

@byronyi

Is this issue present? If yes, please update the status with above suggestion. Is no, could you close it?

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No

[byronyi]

The nGraph build option seems to be removed and migrated to a separate package. Closing for now.

[NeoZhangJianyu]

@byronyi

Thank you very much!