search

opencv / opencv

Open Source Computer Vision Library
https://opencv.org
Apache License 2.0
75.69k stars 55.61k forks source link

Cannot read a tensorflow model #25437

Open LaurentBerger opened 2 weeks ago

LaurentBerger commented 2 weeks ago 
### System Information

General configuration for OpenCV 5.0.0-pre =====================================
  Version control:               4.9.0-603-g0e1d326ed0

  Extra modules:
    Location (extra):            C:/lib/opencv_contrib/modules
    Version control (extra):     4.9.0-121-g542e22ae

  Platform:
    Timestamp:                   2024-03-11T10:55:26Z
    Host:                        Windows 10.0.22631 AMD64
    CMake:                       3.26.1
    CMake generator:             Visual Studio 17 2022
    CMake build tool:            C:/Program Files/Microsoft Visual Studio/2022/Community/MSBuild/Current/Bin/amd64/MSBuild.exe
    MSVC:                        1935
    Configuration:               Debug Release

  CPU/HW features:
    Baseline:                    SSE SSE2 SSE3
      requested:                 SSE3
    Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
      requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
      SSE4_1 (19 files):         + SSSE3 SSE4_1
      SSE4_2 (2 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
      FP16 (1 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
      AVX (9 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
      AVX2 (40 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
      AVX512_SKX (8 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX

  C/C++:
    Built as dynamic libs?:      YES
    C++ standard:                11
    C++ Compiler:                C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.35.32215/bin/Hostx64/x64/cl.exe  (ver 19.35.32215.0)
    C++ flags (Release):         /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP  /O2 /Ob2 /DNDEBUG
    C++ flags (Debug):           /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP  /Zi /Ob0 /Od /RTC1
    C Compiler:                  C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.35.32215/bin/Hostx64/x64/cl.exe
    C flags (Release):           /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /MP   /O2 /Ob2 /DNDEBUG
    C flags (Debug):             /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /MP /Zi /Ob0 /Od /RTC1
    Linker flags (Release):      /machine:x64  /INCREMENTAL:NO
    Linker flags (Debug):        /machine:x64  /debug /INCREMENTAL
    ccache:                      NO
    Precompiled headers:         YES
    Extra dependencies:          cudart_static.lib nppc.lib nppial.lib nppicc.lib nppidei.lib nppif.lib nppig.lib nppim.lib nppist.lib nppisu.lib nppitc.lib npps.lib cublas.lib cudnn.lib cufft.lib -LIBPATH:C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/lib/x64
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 3d alphamat bgsegm bioinspired calib ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform java line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency sfm shape signal stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab viz wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto xstereo
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 cannops cvv freetype hdf julia matlab ovis
    Applications:                tests perf_tests examples apps
    Documentation:               doxygen python javadoc
    Non-free algorithms:         YES

  Windows RT support:            NO

  GUI:                           WIN32UI
    Win32 UI:                    YES
    OpenGL support:              YES (opengl32 glu32)
    VTK support:                 YES (ver 9.2.5)
    Built-in Unicode font:       YES

  Media I/O:
    ZLib:                        optimized C:/install/zlib/lib/zlib.lib debug C:/install/zlib/lib/zlibd.lib (ver 1.2.13)
    JPEG:                        build-libjpeg-turbo (ver 2.1.3-62)
      SIMD Support Request:      YES
      SIMD Support:              NO
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         optimized C:/install/libpng/lib/libpng16.lib debug C:/install/libpng/lib/libpng16d.lib (ver 1.6.40.git)
    TIFF:                        build (ver 42 - 4.6.0)
    JPEG 2000:                   build (ver 2.5.0)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES (prebuilt binaries)
      avcodec:                   YES (58.134.100)
      avformat:                  YES (58.76.100)
      avutil:                    YES (56.70.100)
      swscale:                   YES (5.9.100)
      avresample:                YES (4.0.0)
    GStreamer:                   NO
    DirectShow:                  YES
    Media Foundation:            YES
      DXVA:                      YES

  Parallel framework:            TBB (ver 2020.3 interface 11103)
  Other third-party libraries:
    Intel IPP:                   2021.10.1 [2021.10.1]
           at:                   C:/lib/build/opencv/3rdparty/ippicv/ippicv_win/icv
    Intel IPP IW:                sources (2021.10.1)
              at:                C:/lib/build/opencv/3rdparty/ippicv/ippicv_win/iw
    Lapack:                      YES (libclapack 3.9.0)
    OpenVINO:                    YES (2022.3.0)
    Eigen:                       YES (ver ..)
    Custom HAL:                  NO
    Protobuf:                    build (3.19.1)
    Flatbuffers:                 builtin/3rdparty (23.5.9)

  NVIDIA CUDA:                   YES (ver 12.1, CUFFT CUBLAS)
    NVIDIA GPU arch:             86
    NVIDIA PTX archs:

  cuDNN:                         YES (ver 8.8.0)

  OpenCL:                        YES (NVD3D11)
    Include path:                C:/lib/opencv/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python 3:
    Interpreter:                 C:/Program Files/Python310/python.exe (ver 3.10.10)
    Libraries:                   optimized C:/Program Files/Python310/libs/python310.lib debug C:/Program Files/Python310/libs/python310_d.lib (ver 3.10.10)
    Limited API:                 NO
    numpy:                       C:/Users/laurent/AppData/Roaming/Python/Python310/site-packages/numpy/core/include (ver 1.23.5)
    install path:                C:/Users/laurent/AppData/Roaming/Python/Python310/site-packages/cv2/python-3.10

  Python (for build):            C:/Program Files/Python310/python.exe

  Java:
    ant:                         C:/apache-ant-1.10.13/bin/ant.bat (ver 1.10.13)
    Java:                        NO
    JNI:                         C:/Program Files/Java/jdk-17/include C:/Program Files/Java/jdk-17/include/win32 C:/Program Files/Java/jdk-17/include
    Java wrappers:               YES (ANT)
    Java tests:                  YES

  Install to:                    C:/install/opencv

Detailed description

I tried to read a tensorflow model and i Have got error : 

[libprotobuf ERROR C:\lib\opencv\3rdparty\protobuf\src\google\protobuf\wire_format_lite.cc:581] String field 'opencv_tensorflow.FunctionDef.Node.ret' contains invalid UTF-8 data when parsing a protocol buffer. Use the 'bytes' type if you intend to send raw bytes.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
cv2.error: OpenCV(5.0.0-pre) C:\lib\opencv\modules\dnn\src\tensorflow\tf_io.cpp:42: error: (-2:Unspecified error) FAILED: ReadProtoFromBinaryFile(param_file, param). Failed to parse GraphDef file: a.pb/saved_model.pb in function 'cv::dnn::ReadTFNetParamsFromBinaryFileOrDie'

onnx model test_pad.zip

model a.pb.zip

Steps to reproduce

Two steps first create onnx model

import numpy as np
import onnx
import onnxsim
import onnx.reference
op = onnx.OperatorSetIdProto()
op.version = 13

onnx_name = "test_pad"

x = np.array( [[10, 11], [12, 13], [14, 15]]).astype(np.int32)

node2 = onnx.helper.make_node(
    "Concat",
    inputs=["inpa", "inpb"],
    outputs=["pad_values"],
    axis=0
)

out_pad = onnx.helper.make_tensor_value_info('out_pad', onnx.TensorProto.INT32, [None, None, None, None])
inpx = onnx.helper.make_tensor_value_info('inpx', onnx.TensorProto.INT32, x.shape)
inpa = onnx.helper.make_tensor_value_info('inpa', onnx.TensorProto.INT32, (2,))
inpb = onnx.helper.make_tensor_value_info('inpb', onnx.TensorProto.INT32, (2,))

node1 = onnx.helper.make_node( "Pad", inputs=["inpx", "pad_values"], outputs=["out_pad"], mode="constant")
graph = onnx.helper.make_graph([node2, node1], onnx_name,  [inpx, inpa, inpb], [out_pad])
onnx_model = onnx.helper.make_model(graph, opset_imports=[op])

feeds = {'inpx': x, 'inpa': np.array([1, 2]), 'inpb': np.array([3, 1])}
sess = onnx.reference.ReferenceEvaluator(onnx_model)
res_onnx = sess.run(["out_pad"], feeds)
print("ONNX result")
print(res_onnx[0])
print("Writting model")
with open(onnx_name + ".onnx", "wb") as f:
    f.write(onnx_model.SerializeToString())

Second step convert onnx model to tensorflow

onnx-tf convert -i test_pad.onnx -o a.pb

Issue submission checklist