search

opencv / opencv

Open Source Computer Vision Library
https://opencv.org
Apache License 2.0
79.29k stars 55.84k forks source link

Python bindings do not respect optional modules #26098

Open allsey87 opened 2 months ago

allsey87 commented 2 months ago

System Information

OpenCV 4.10.0

Detailed description

When building OpenCV with Python support, it seems that I am forced to compile a bunch of modules that I do not want in order to generate the Python bindings. So far, I have been forced to compile imgproc, features2d, gapi, and dnn.

Steps to reproduce

When running CMake, define the BUILD_LIST as core,python_bindings_generator,python_tests,python3 and watch it fail with SymbolNotFound and TypeResolutionFailed errors.

Issue submission checklist

asmorkalov commented 2 months ago

@allsey87 There several "hard-coded" dependencies in bindings. Could you provide build log and configuration details?

molar commented 2 months ago

Hello, we are encountering the same issue, here is the trimmed output of configure

$ cmake -B build -G Ninja -D CMAKE_EXPORT_COMPILE_COMMANDS=1 \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
-DBUILD_LIST="core,features2d,imgproc,imgcodecs,3dcalib,video,videoio,flann,python3"

...

-- General configuration for OpenCV 4.10.0 =====================================
--   Version control:               4.10.0
-- 
--   Platform:
--     Timestamp:                   2024-09-25T07:43:58Z
--     Host:                        Linux 5.15.0-122-generic x86_64
--     CMake:                       3.22.1
--     CMake generator:             Ninja
--     CMake build tool:            /usr/bin/ninja
--     Configuration:               Release
-- 
--   CPU/HW features:
--     Baseline:                    SSE SSE2 SSE3
--       requested:                 SSE3
--     Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
--       requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
--       SSE4_1 (14 files):         + SSSE3 SSE4_1
--       SSE4_2 (1 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
--       FP16 (0 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
--       AVX (3 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
--       AVX2 (27 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
--       AVX512_SKX (3 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX
-- 
--   C/C++:
--     Built as dynamic libs?:      YES
--     C++ standard:                11
--     C++ Compiler:                /usr/bin/c++  (ver 11.4.0)
--     C++ flags (Release):         -fsigned-char -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshado
w -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ff
unction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
--     C++ flags (Debug):           -fsigned-char -W -Wall -Wreturn-type -Wnon-virtual-dtor -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshado
w -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ff
unction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
--     C Compiler:                  /usr/bin/cc
--     C flags (Release):           -fsigned-char -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -
Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse
 -msse2 -msse3 -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
--     C flags (Debug):             -fsigned-char -W -Wall -Wreturn-type -Waddress -Wsequence-point -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -
Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse
 -msse2 -msse3 -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
--     Linker flags (Release):      -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a   -Wl,--gc-sections -Wl,--as-needed -Wl,--no-undefined  
--     Linker flags (Debug):        -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a   -Wl,--gc-sections -Wl,--as-needed -Wl,--no-undefined  
--     ccache:                      YES
--     Precompiled headers:         NO
--     Extra dependencies:          dl m pthread rt
--     3rdparty dependencies:
-- 
--   OpenCV modules:
--     To be built:                 core features2d flann imgcodecs imgproc python3 video videoio
--     Disabled:                    world
--     Disabled by dependency:      calib3d dnn gapi highgui java java_bindings_generator js_bindings_generator ml objc_bindings_generator objdetect photo python_tests stitching ts
--     Unavailable:                 python2
--     Applications:                apps
--     Documentation:               NO
--     Non-free algorithms:         NO
-- 
--   GUI: 
--     GTK+:                        YES (ver 3.24.33)
--       GThread :                  YES (ver 2.72.4)
--       GtkGlExt:                  NO
--     VTK support:                 NO
-- 
--   Media I/O: 
--     ZLib:                        /usr/lib/x86_64-linux-gnu/libz.so (ver 1.2.11)
--     JPEG:                        /usr/lib/x86_64-linux-gnu/libjpeg.so (ver 80)
--     WEBP:                        build (ver encoder: 0x020f)
--     PNG:                         /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.6.37)
--     TIFF:                        /usr/lib/x86_64-linux-gnu/libtiff.so (ver 42 / 4.3.0)
--     JPEG 2000:                   build (ver 2.5.0)
--     OpenEXR:                     build (ver 2.3.0)
--     HDR:                         YES
--     SUNRASTER:                   YES
--     PXM:                         YES
--     PFM:                         YES
-- 
--   Video I/O:
--     DC1394:                      NO
--     FFMPEG:                      NO
--       avcodec:                   NO
--       avformat:                  NO
--       avutil:                    NO
--       swscale:                   NO
--       avresample:                NO
--     GStreamer:                   NO
--     v4l/v4l2:                    YES (linux/videodev2.h)
-- 
--   Parallel framework:            pthreads
-- 
--   Trace:                         YES (with Intel ITT)
-- 
--   Other third-party libraries:
--     Intel IPP:                   2021.11.0 [2021.11.0]
--            at:                   /home/mirmla/work/opencv/build/3rdparty/ippicv/ippicv_lnx/icv
--     Intel IPP IW:                sources (2021.11.0)
--               at:                /home/mirmla/work/opencv/build/3rdparty/ippicv/ippicv_lnx/iw
--     VA:                          NO
--     Lapack:                      NO
--     Eigen:                       NO
--     Custom HAL:                  NO
--     Protobuf:                    build (3.19.1)
--     Flatbuffers:                 builtin/3rdparty (23.5.9)
-- 
--   OpenCL:                        YES (no extra features)
--     Include path:                /home/mirmla/work/opencv/3rdparty/include/opencl/1.2
--     Link libraries:              Dynamic load
-- 
--   Python 3:
--     Interpreter:                 /usr/bin/python3 (ver 3.10.12)
--     Libraries:                   /usr/lib/x86_64-linux-gnu/libpython3.10.so (ver 3.10.12)
--     Limited API:                 NO
--     numpy:                       /home/mirmla/.local/lib/python3.10/site-packages/numpy/core/include (ver 1.25.2)
--     install path:                lib/python3.10/dist-packages/cv2/python-3.10
-- 
--   Python (for build):            /usr/bin/python3
-- 
--   Java:                          
--     ant:                         NO
--     Java:                        YES (ver 18.0.2)
--     JNI:                         /usr/lib/jvm/default-java/include /usr/lib/jvm/default-java/include/linux /usr/lib/jvm/default-java/include
--     Java wrappers:               NO
--     Java tests:                  NO
-- 
--   Install to:                    /usr/local
-- -----------------------------------------------------------------
-- 
-- Configuring done
-- Generating done
-- Build files have been written to: /home/mirmla/work/opencv/build
~/w/opencv 4.10.0• 9.1s ❱ cmake --build build --
[285/611] Generate files for Python bindings and documentation
/home/mirmla/work/opencv/modules/python/src2/typing_stubs_generator.py:52: UserWarning: Typing stubs generation has failed.
Traceback (most recent call last):
  File "/home/mirmla/work/opencv/modules/python/src2/typing_stubs_generation/nodes/type_node.py", line 291, in resolve
    self.value.resolve(root)
  File "/home/mirmla/work/opencv/modules/python/src2/typing_stubs_generation/nodes/type_node.py", line 626, in resolve
    raise TypeResolutionError(
typing_stubs_generation.nodes.type_node.TypeResolutionError: Failed to resolve one of "_typing.Union[cv2.typing.Scalar, GMat, GOpaqueT, GArrayT]" items. Errors: ['Failed to resolve "GMat" exposed as "GMat"', 'Fai
led to resolve "GOpaqueT" exposed as "GOpaqueT"', 'Failed to resolve "GArrayT" exposed as "GArrayT"']

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/mirmla/work/opencv/modules/python/src2/typing_stubs_generator.py", line 49, in wrapped_func
    ret_type = func(*args, **kwargs)
  File "/home/mirmla/work/opencv/modules/python/src2/typing_stubs_generator.py", line 148, in _generate
    generate_typing_stubs(self.cv_root, output_path)
  File "/home/mirmla/work/opencv/modules/python/src2/typing_stubs_generation/generation.py", line 91, in generate_typing_stubs
    _generate_typing_module(root, output_path)
  File "/home/mirmla/work/opencv/modules/python/src2/typing_stubs_generation/generation.py", line 794, in _generate_typing_module
    node.resolve(root)
  File "/home/mirmla/work/opencv/modules/python/src2/typing_stubs_generation/nodes/type_node.py", line 293, in resolve
    raise TypeResolutionError(
typing_stubs_generation.nodes.type_node.TypeResolutionError: Failed to resolve alias "GProtoArg" exposed as "GProtoArg"

  warnings.warn(
Note: Class cv::Feature2D has more than 1 base class (not supported by Python C extensions)
      Bases:  cv::Algorithm, cv::class, cv::Feature2D, cv::Algorithm
      Only the first base class will be used

We are working around it by applied the following patch 

diff --git a/modules/python/src2/typing_stubs_generation/predefined_types.py b/modules/python/src2/typing_stubs_generation/predefined_types.py
index f5ba0bc29e..ab0d631c70 100644
--- a/modules/python/src2/typing_stubs_generation/predefined_types.py
+++ b/modules/python/src2/typing_stubs_generation/predefined_types.py
@@ -38,10 +38,13 @@ _PREDEFINED_TYPES = (
         "NumPyArrayNumeric",
         dtype="numpy.integer[_typing.Any] | numpy.floating[_typing.Any]"
     ),
-    ConditionalAliasTypeNode.numpy_array_("NumPyArrayFloat32", dtype="numpy.float32"),
-    ConditionalAliasTypeNode.numpy_array_("NumPyArrayFloat64", dtype="numpy.float64"),
+    ConditionalAliasTypeNode.numpy_array_(
+        "NumPyArrayFloat32", dtype="numpy.float32"),
+    ConditionalAliasTypeNode.numpy_array_(
+        "NumPyArrayFloat64", dtype="numpy.float64"),
     NoneTypeNode("void"),
-    AliasTypeNode.int_("void*", "IntPointer", "Represents an arbitrary pointer"),
+    AliasTypeNode.int_("void*", "IntPointer",
+                       "Represents an arbitrary pointer"),
     AliasTypeNode.union_(
         "Mat",
         items=(ASTNodeTypeNode("Mat", module_name="cv2.mat_wrapper"),
@@ -117,38 +120,6 @@ _PREDEFINED_TYPES = (
                          export_name="DescriptorExtractor"),
     AliasTypeNode.class_("FeatureExtractor", "Feature2D",
                          export_name="FeatureExtractor"),
-    AliasTypeNode.union_("GProtoArg",
-                         items=(AliasRefTypeNode("Scalar"),
-                                ASTNodeTypeNode("GMat"),
-                                ASTNodeTypeNode("GOpaqueT"),
-                                ASTNodeTypeNode("GArrayT"))),
-    SequenceTypeNode("GProtoArgs", AliasRefTypeNode("GProtoArg")),
-    AliasTypeNode.sequence_("GProtoInputArgs", AliasRefTypeNode("GProtoArg")),
-    AliasTypeNode.sequence_("GProtoOutputArgs", AliasRefTypeNode("GProtoArg")),
-    AliasTypeNode.union_(
-        "GRunArg",
-        items=(AliasRefTypeNode("Mat", "MatLike"),
-               AliasRefTypeNode("Scalar"),
-               ASTNodeTypeNode("GOpaqueT"),
-               ASTNodeTypeNode("GArrayT"),
-               SequenceTypeNode("GRunArg", AnyTypeNode("GRunArg")),
-               NoneTypeNode("GRunArg"))
-    ),
-    AliasTypeNode.optional_("GOptRunArg", AliasRefTypeNode("GRunArg")),
-    AliasTypeNode.union_("GMetaArg",
-                         items=(ASTNodeTypeNode("GMat"),
-                                AliasRefTypeNode("Scalar"),
-                                ASTNodeTypeNode("GOpaqueT"),
-                                ASTNodeTypeNode("GArrayT"))),
-    AliasTypeNode.union_("Prim",
-                         items=(ASTNodeTypeNode("gapi.wip.draw.Text"),
-                                ASTNodeTypeNode("gapi.wip.draw.Circle"),
-                                ASTNodeTypeNode("gapi.wip.draw.Image"),
-                                ASTNodeTypeNode("gapi.wip.draw.Line"),
-                                ASTNodeTypeNode("gapi.wip.draw.Rect"),
-                                ASTNodeTypeNode("gapi.wip.draw.Mosaic"),
-                                ASTNodeTypeNode("gapi.wip.draw.Poly"))),
-    SequenceTypeNode("Prims", AliasRefTypeNode("Prim")),
     AliasTypeNode.array_ref_("Matx33f",
                              array_ref_name="NumPyArrayFloat32",
                              shape=(3, 3),
@@ -167,33 +138,6 @@ _PREDEFINED_TYPES = (
                              dtype="numpy.float64"),
     NDArrayTypeNode("vector<uchar>", dtype="numpy.uint8"),
     NDArrayTypeNode("vector_uchar", dtype="numpy.uint8"),
-    TupleTypeNode("GMat2", items=(ASTNodeTypeNode("GMat"),
-                                  ASTNodeTypeNode("GMat"))),
-    ASTNodeTypeNode("GOpaque", "GOpaqueT"),
-    ASTNodeTypeNode("GArray", "GArrayT"),
-    AliasTypeNode.union_("GTypeInfo",
-                         items=(ASTNodeTypeNode("GMat"),
-                                AliasRefTypeNode("Scalar"),
-                                ASTNodeTypeNode("GOpaqueT"),
-                                ASTNodeTypeNode("GArrayT"))),
-    SequenceTypeNode("GCompileArgs", ASTNodeTypeNode("GCompileArg")),
-    SequenceTypeNode("GTypesInfo", AliasRefTypeNode("GTypeInfo")),
-    SequenceTypeNode("GRunArgs", AliasRefTypeNode("GRunArg")),
-    SequenceTypeNode("GMetaArgs", AliasRefTypeNode("GMetaArg")),
-    SequenceTypeNode("GOptRunArgs", AliasRefTypeNode("GOptRunArg")),
-    AliasTypeNode.callable_(
-        "detail_ExtractArgsCallback",
-        arg_types=SequenceTypeNode("GTypesInfo", AliasRefTypeNode("GTypeInfo")),
-        ret_type=SequenceTypeNode("GRunArgs", AliasRefTypeNode("GRunArg")),
-        export_name="ExtractArgsCallback"
-    ),
-    AliasTypeNode.callable_(
-        "detail_ExtractMetaCallback",
-        arg_types=SequenceTypeNode("GTypesInfo", AliasRefTypeNode("GTypeInfo")),
-        ret_type=SequenceTypeNode("GMetaArgs", AliasRefTypeNode("GMetaArg")),
-        export_name="ExtractMetaCallback"
-    ),
-    AliasTypeNode.class_("LayerId", "DictValue"),
     PrimitiveTypeNode.int_("cvflann_flann_distance_t"),
     PrimitiveTypeNode.int_("flann_flann_distance_t"),
     PrimitiveTypeNode.int_("cvflann_flann_algorithm_t"),

but we would like to help with a more sustainable solution. Let me know if you need more logs.