TFLite with OpenCV Issue

lahrims commented 7 months ago

System Information

Python: 3.10.13 Tensorflow-macos : 2.15.0 Opencv-python: 4.9.0.80 TFLite:2.10.0 TFLite-support: 0.1.0a1

Detailed description

I am trying to deploy OpenCV with TFLite (Experimental), its classification model (LeNet5), i usually use ONNX. The TFLite test works fine until i am using the TFLite internal interpreter. But it fails with [ERROR:0@2.615] global tflite_importer.cpp:246 populateNet DNN/TFLite: Problem during import of operator [CAST]:(model/rescale_uint8/Cast) (10/22). Exception: OpenCV(4.9.0) /xperience/GHA-OpenCV-Python2/_work/opencv-python/opencv-python/opencv/modules/dnn/src/tflite/tflite_importer.cpp:238: error: (-213:The function/feature is not implemented) Unsupported operator type CAST in function 'populateNet' when i try it with net = cv2.dnn.readNetFromTFLite(name_base+".tflite").

For Context: This is how i converted the model to TFLite


converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_types = [tf.float16]
tflite_model = converter.convert()
# Save the converted model to a file
temp_model_file = name_base + ".tflite"
with open(temp_model_file,'wb') as f:
   f.write(tflite_model)

Steps to reproduce

Try reproducing with a basic Lenet5 classifier and testing it out witth cv2.dnn.readNetFromTFLite

Issue submission checklist

[X] I report the issue, it's not a question
[X] I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
[X] I updated to the latest OpenCV version and the issue is still there

LaurentBerger commented 7 months ago

I haven't got same error.

[ERROR:0@36.472] global tflite_importer.cpp:246 cv::dnn::dnn5_v20231225::TFLiteImporter::populateNet DNN/TFLite: Problem during import of operator [FULLY_CONNECTED]:(sequential/dense/MatMul;sequential/dense/Relu;sequential/dense/BiasAdd) (15/19). Exception: OpenCV(5.0.0-pre) C:\lib\opencv\modules\dnn\src\tflite\tflite_importer.cpp:238: error: (-213:The function/feature is not implemented) Unsupported operator type FULLY_CONNECTED in function 'cv::dnn::dnn5_v20231225::TFLiteImporter::populateNet'

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
cv2.error: OpenCV(5.0.0-pre) C:\lib\opencv\modules\dnn\src\tflite\tflite_importer.cpp:238: error: (-213:The function/feature is not implemented) Unsupported operator type FULLY_CONNECTED in function 'cv::dnn::dnn5_v20231225::TFLiteImporter::populateNet'

Can you post your tflite model? My model is lenet5.zip

lahrims commented 7 months ago

Sure, Here is my model zip demo_model.zip

LaurentBerger commented 7 months ago

Two layers seems not implemented in this model CAST and FullyConnected

dkurt commented 7 months ago

working on FC layer in https://github.com/opencv/opencv/pull/25273. Need more time to cover Cast (currently fully implemented on 5.x but not in 4.x)

LaurentBerger commented 7 months ago

(currently fully implemented on 5.x but not in 4.x)

not sure :

General configuration for OpenCV 5.0.0-pre =====================================
  Version control:               4.9.0-490-g55426ee195

  Extra modules:
    Location (extra):            C:/lib/opencv_contrib/modules
    Version control (extra):     4.9.0-89-g3f609aa2

  Platform:
    Timestamp:                   2024-03-11T10:55:26Z
    Host:                        Windows 10.0.22631 AMD64
    CMake:                       3.26.1
    CMake generator:             Visual Studio 17 2022
    CMake build tool:            C:/Program Files/Microsoft Visual Studio/2022/Community/MSBuild/Current/Bin/amd64/MSBuild.exe
    MSVC:                        1935
    Configuration:               Debug Release

  CPU/HW features:
    Baseline:                    SSE SSE2 SSE3
      requested:                 SSE3
    Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
      requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
      SSE4_1 (19 files):         + SSSE3 SSE4_1
      SSE4_2 (2 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
      FP16 (1 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
      AVX (9 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
      AVX2 (40 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
      AVX512_SKX (8 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX

  C/C++:
    Built as dynamic libs?:      YES
    C++ standard:                11
    C++ Compiler:                C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.35.32215/bin/Hostx64/x64/cl.exe  (ver 19.35.32215.0)
    C++ flags (Release):         /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP  /O2 /Ob2 /DNDEBUG
    C++ flags (Debug):           /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP  /Zi /Ob0 /Od /RTC1
    C Compiler:                  C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.35.32215/bin/Hostx64/x64/cl.exe
    C flags (Release):           /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /MP   /O2 /Ob2 /DNDEBUG
    C flags (Debug):             /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /MP /Zi /Ob0 /Od /RTC1
    Linker flags (Release):      /machine:x64  /INCREMENTAL:NO
    Linker flags (Debug):        /machine:x64  /debug /INCREMENTAL
    ccache:                      NO
    Precompiled headers:         YES
    Extra dependencies:          cudart_static.lib nppc.lib nppial.lib nppicc.lib nppidei.lib nppif.lib nppig.lib nppim.lib nppist.lib nppisu.lib nppitc.lib npps.lib cublas.lib cudnn.lib cufft.lib -LIBPATH:C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/lib/x64
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 3d alphamat aruco bgsegm bioinspired calib ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform java line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency sfm shape signal stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab viz wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto xstereo
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 cannops cvv freetype hdf julia matlab ovis
    Applications:                tests perf_tests examples apps
    Documentation:               doxygen python javadoc
    Non-free algorithms:         YES

  Windows RT support:            NO

  GUI:                           WIN32UI
    Win32 UI:                    YES
    OpenGL support:              YES (opengl32 glu32)
    VTK support:                 YES (ver 9.2.5)
    Built-in Unicode font:       YES

  Media I/O:
    ZLib:                        optimized C:/install/zlib/lib/zlib.lib debug C:/install/zlib/lib/zlibd.lib (ver 1.2.13)
    JPEG:                        build-libjpeg-turbo (ver 2.1.3-62)
      SIMD Support Request:      YES
      SIMD Support:              NO
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         optimized C:/install/libpng/lib/libpng16.lib debug C:/install/libpng/lib/libpng16d.lib (ver 1.6.40.git)
    TIFF:                        build (ver 42 - 4.2.0)
    JPEG 2000:                   build (ver 2.5.0)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES (prebuilt binaries)
      avcodec:                   YES (58.134.100)
      avformat:                  YES (58.76.100)
      avutil:                    YES (56.70.100)
      swscale:                   YES (5.9.100)
      avresample:                YES (4.0.0)
    GStreamer:                   NO
    DirectShow:                  YES
    Media Foundation:            YES
      DXVA:                      YES

  Parallel framework:            TBB (ver 2020.3 interface 11103)

  Other third-party libraries:
    Intel IPP:                   2021.10.1 [2021.10.1]
           at:                   C:/lib/build/opencv/3rdparty/ippicv/ippicv_win/icv
    Intel IPP IW:                sources (2021.10.1)
              at:                C:/lib/build/opencv/3rdparty/ippicv/ippicv_win/iw
    Lapack:                      YES (libclapack 3.9.0)
    OpenVINO:                    YES (2022.3.0)
    Eigen:                       YES (ver ..)
    Custom HAL:                  NO
    Protobuf:                    build (3.19.1)
    Flatbuffers:                 builtin/3rdparty (23.5.9)

  NVIDIA CUDA:                   YES (ver 12.1, CUFFT CUBLAS)
    NVIDIA GPU arch:             86
    NVIDIA PTX archs:

  cuDNN:                         YES (ver 8.8.0)

  OpenCL:                        YES (NVD3D11)
    Include path:                C:/lib/opencv/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python 3:
    Interpreter:                 C:/Program Files/Python310/python.exe (ver 3.10.10)
    Libraries:                   optimized C:/Program Files/Python310/libs/python310.lib debug C:/Program Files/Python310/libs/python310_d.lib (ver 3.10.10)
    Limited API:                 NO
    numpy:                       C:/Users/laurent/AppData/Roaming/Python/Python310/site-packages/numpy/core/include (ver 1.23.5)
    install path:                C:/Users/laurent/AppData/Roaming/Python/Python310/site-packages/cv2/python-3.10

  Python (for build):            C:/Program Files/Python310/python.exe

  Java:
    ant:                         C:/apache-ant-1.10.13/bin/ant.bat (ver 1.10.13)
    Java:                        NO
    JNI:                         C:/Program Files/Java/jdk-17/include C:/Program Files/Java/jdk-17/include/win32 C:/Program Files/Java/jdk-17/include
    Java wrappers:               YES (ANT)
    Java tests:                  YES

  Install to:                    C:/install/opencv
-----------------------------------------------------------------

OpenCV(5.0.0-pre) Error: The function/feature is not implemented (Unsupported operator type CAST) in cv::dnn::dnn5_v20231225::TFLiteImporter::populateNet, file C:\lib\opencv\modules\dnn\src\tflite\tflite_importer.cpp, line 238
[ERROR:0@5.694] global tflite_importer.cpp:246 cv::dnn::dnn5_v20231225::TFLiteImporter::populateNet DNN/TFLite: Problem during import of operator [CAST]:(model/rescale_uint8/Cast) (10/22). Exception: OpenCV(5.0.0-pre) C:\lib\opencv\modules\dnn\src\tflite\tflite_importer.cpp:238: error: (-213:The function/feature is not implemented) Unsupported operator type CAST in function 'cv::dnn::dnn5_v20231225::TFLiteImporter::populateNet'

dkurt commented 7 months ago

@LaurentBerger, I mean the layer itself, not TFLite importer: https://github.com/opencv/opencv/blob/5.x/modules/dnn/src/layers/cast_layer.cpp

opencv / opencv