Open gafrom opened 3 years ago
Hi @gafrom, have you figure it out? This work around is good or not? I'm facing the same issue with my jetson nano.
Hi @HungDuong1998 , sorry, did not have capacity to dive into this. Back then I ended up running it without lapack, thought to switch to something GPUish like Jetson nano in the near future as I thought this issue is related to CPU mode only - apparently not.
Re if running without lapack is good or not - I do not have any data to confirm how fast it is with lapack, so cannot say.
I'm upgrading opencv and linux kernel today and plan to spend some time on this issue next week if it persists, will keep updates on this thread. If you have any ideas how to solve it - please share.
GPU has its acceleration advantages to overcome the lackpack missing but you need to configure it manually as it is not yet supported by default. If you can do, please do it so I can see too haha, the documentation of configuration is not yet released and currently I'm searching the configurations in the issue section, which is painful haha.
So, I figured out what was happening. I was missing this flag:
"BUILD_SHARED_LIBS": "ON"
Basically what was happening is that I turned on WITH_LAPACK
, but after compilation opencv could not find this shared library (for some reason although it was installed with apt-get) hence ctypes
could not resolve the method name.
However, according to the comments one cannot use both shared and static (with shared being a default):
# Note: passing both BUILD_SHARED_LIBS=ON and BUILD_STATIC_LIBS=ON to cmake
# still only builds the shared libraries, so we have to choose one or the
# other. We build shared libraries by default, but this variable can be used
# to switch to static libraries.
OPENCV_SHARED_LIBS = False
And if you try changing OPENCV_SHARED_LIBS
to True
, it won't work :) Actually I tried, but then realized that this line of code gets replaced on every build according to the options passed to setup.py
.
So, in the end, I managed to use static libs by executing this:
python3 setup.py gen_protos && python3 setup.py bdist_wheel build --link-opencv
Now, I'm checking how much improvement I got with LAPACK. Will comment on this later.
So, in the daytime (the slowest inference, as images contain a lot of bright pixels) it gave me 2X improvement. (9.92 fps with LAPACK vs 5.4 fps without LAPACK). Will report on evening results later.
Thanks for sharing, on trying to reproduce your result with:
python3 setup.py gen_protos && python3 setup.py bdist_wheel build --link-opencv
I encounter this error:
fatal error: opencv2/core/version.hpp: No such file or directory
maybe it's because of arm64 architecture that the opencv files are in different places. What do you think I should do? modifying:
"BUILD_SHARED_LIBS": "ON" if OPENCV_SHARED_LIBS else "OFF",
to
"BUILD_SHARED_LIBS": "ON",
?
You can try enabling shared libs only in this place, but I reckon it won't work as there is quite a lot of things tangled onto this flag.
Can you paste more about your error? At which step of build process does it break?
Modifying
"BUILD_SHARED_LIBS": "ON" if OPENCV_SHARED_LIBS else "OFF",
to
"BUILD_SHARED_LIBS": "ON",
Indeed made the build success somehow. But the inference speed is the same on using gpu with or without lapack (~30fps) . I think GPU version doesn't really need lapack.
Now back to the error log, this is the full log on changing
OPENCV_SHARED_LIBS = False
-> OPENCV_SHARED_LIBS = True
and build with:
python3 setup.py gen_protos && python3 setup.py bdist_wheel build --link-opencv
(Sorry, kinda long because I'm not yet used to github commenting section, don't know how to make collapse or scroll section)
python3 setup.py gen_protos && python3 setup.py bdist_wheel build --link-opencv running gen_protos adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/calculators/init.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/modules/init.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/modules/holistic_landmark/init.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/modules/objectron/init.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/framework/init.py generating proto file: mediapipe/framework/calculator_pb2.py generating proto file: mediapipe/framework/calculator_options_pb2.py generating proto file: mediapipe/framework/test_calculators_pb2.py generating proto file: mediapipe/framework/packet_factory_pb2.py generating proto file: mediapipe/framework/packet_generator_pb2.py generating proto file: mediapipe/framework/mediapipe_options_pb2.py generating proto file: mediapipe/framework/status_handler_pb2.py generating proto file: mediapipe/framework/stream_handler_pb2.py generating proto file: mediapipe/framework/thread_pool_executor_pb2.py generating proto file: mediapipe/framework/calculator_profile_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/framework/tool/init.py generating proto file: mediapipe/framework/tool/node_chain_subgraph_pb2.py generating proto file: mediapipe/framework/tool/source_pb2.py generating proto file: mediapipe/framework/tool/switch_container_pb2.py generating proto file: mediapipe/framework/tool/calculator_graph_template_pb2.py generating proto file: mediapipe/framework/tool/packet_generator_wrapper_calculator_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/framework/deps/init.py generating proto file: mediapipe/framework/deps/proto_descriptor_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/framework/stream_handler/init.py generating proto file: mediapipe/framework/stream_handler/timestamp_align_input_stream_handler_pb2.py generating proto file: mediapipe/framework/stream_handler/sync_set_input_stream_handler_pb2.py generating proto file: mediapipe/framework/stream_handler/default_input_stream_handler_pb2.py generating proto file: mediapipe/framework/stream_handler/fixed_size_input_stream_handler_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/framework/formats/init.py generating proto file: mediapipe/framework/formats/image_format_pb2.py generating proto file: mediapipe/framework/formats/image_file_properties_pb2.py generating proto file: mediapipe/framework/formats/matrix_data_pb2.py generating proto file: mediapipe/framework/formats/landmark_pb2.py generating proto file: mediapipe/framework/formats/classification_pb2.py generating proto file: mediapipe/framework/formats/rect_pb2.py generating proto file: mediapipe/framework/formats/location_data_pb2.py generating proto file: mediapipe/framework/formats/detection_pb2.py generating proto file: mediapipe/framework/formats/time_series_header_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/framework/formats/motion/init.py generating proto file: mediapipe/framework/formats/motion/optical_flow_field_data_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/framework/formats/annotation/init.py generating proto file: mediapipe/framework/formats/annotation/rasterization_pb2.py generating proto file: mediapipe/framework/formats/annotation/locus_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/framework/formats/object_detection/init.py generating proto file: mediapipe/framework/formats/object_detection/anchor_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/calculators/internal/init.py generating proto file: mediapipe/calculators/internal/callback_packet_calculator_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/calculators/util/init.py generating proto file: mediapipe/calculators/util/detections_to_render_data_calculator_pb2.py generating proto file: mediapipe/calculators/util/rect_to_render_scale_calculator_pb2.py generating proto file: mediapipe/calculators/util/labels_to_render_data_calculator_pb2.py generating proto file: mediapipe/calculators/util/refine_landmarks_from_heatmap_calculator_pb2.py generating proto file: mediapipe/calculators/util/packet_frequency_calculator_pb2.py generating proto file: mediapipe/calculators/util/timed_box_list_id_to_label_calculator_pb2.py generating proto file: mediapipe/calculators/util/non_max_suppression_calculator_pb2.py generating proto file: mediapipe/calculators/util/landmarks_to_floats_calculator_pb2.py generating proto file: mediapipe/calculators/util/detections_to_rects_calculator_pb2.py generating proto file: mediapipe/calculators/util/visibility_smoothing_calculator_pb2.py generating proto file: mediapipe/calculators/util/packet_latency_calculator_pb2.py generating proto file: mediapipe/calculators/util/annotation_overlay_calculator_pb2.py generating proto file: mediapipe/calculators/util/top_k_scores_calculator_pb2.py generating proto file: mediapipe/calculators/util/landmark_projection_calculator_pb2.py generating proto file: mediapipe/calculators/util/association_calculator_pb2.py generating proto file: mediapipe/calculators/util/landmarks_to_detection_calculator_pb2.py generating proto file: mediapipe/calculators/util/logic_calculator_pb2.py generating proto file: mediapipe/calculators/util/packet_frequency_pb2.py generating proto file: mediapipe/calculators/util/visibility_copy_calculator_pb2.py generating proto file: mediapipe/calculators/util/rect_to_render_data_calculator_pb2.py generating proto file: mediapipe/calculators/util/landmarks_to_render_data_calculator_pb2.py generating proto file: mediapipe/calculators/util/landmarks_smoothing_calculator_pb2.py generating proto file: mediapipe/calculators/util/detection_label_id_to_text_calculator_pb2.py generating proto file: mediapipe/calculators/util/local_file_contents_calculator_pb2.py generating proto file: mediapipe/calculators/util/thresholding_calculator_pb2.py generating proto file: mediapipe/calculators/util/latency_pb2.py generating proto file: mediapipe/calculators/util/rect_transformation_calculator_pb2.py generating proto file: mediapipe/calculators/util/timed_box_list_to_render_data_calculator_pb2.py generating proto file: mediapipe/calculators/util/collection_has_min_size_calculator_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/calculators/video/init.py generating proto file: mediapipe/calculators/video/flow_packager_calculator_pb2.py generating proto file: mediapipe/calculators/video/box_detector_calculator_pb2.py generating proto file: mediapipe/calculators/video/video_pre_stream_calculator_pb2.py generating proto file: mediapipe/calculators/video/motion_analysis_calculator_pb2.py generating proto file: mediapipe/calculators/video/flow_to_image_calculator_pb2.py generating proto file: mediapipe/calculators/video/tracked_detection_manager_calculator_pb2.py generating proto file: mediapipe/calculators/video/box_tracker_calculator_pb2.py generating proto file: mediapipe/calculators/video/opencv_video_encoder_calculator_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/calculators/video/tool/init.py generating proto file: mediapipe/calculators/video/tool/flow_quantizer_model_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/calculators/audio/init.py generating proto file: mediapipe/calculators/audio/rational_factor_resample_calculator_pb2.py generating proto file: mediapipe/calculators/audio/mfcc_mel_calculators_pb2.py generating proto file: mediapipe/calculators/audio/stabilized_log_calculator_pb2.py generating proto file: mediapipe/calculators/audio/time_series_framer_calculator_pb2.py generating proto file: mediapipe/calculators/audio/spectrogram_calculator_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/calculators/core/init.py generating proto file: mediapipe/calculators/core/constant_side_packet_calculator_pb2.py generating proto file: mediapipe/calculators/core/clip_vector_size_calculator_pb2.py generating proto file: mediapipe/calculators/core/quantize_float_vector_calculator_pb2.py generating proto file: mediapipe/calculators/core/concatenate_vector_calculator_pb2.py generating proto file: mediapipe/calculators/core/flow_limiter_calculator_pb2.py generating proto file: mediapipe/calculators/core/packet_cloner_calculator_pb2.py generating proto file: mediapipe/calculators/core/graph_profile_calculator_pb2.py generating proto file: mediapipe/calculators/core/sequence_shift_calculator_pb2.py generating proto file: mediapipe/calculators/core/dequantize_byte_array_calculator_pb2.py generating proto file: mediapipe/calculators/core/packet_thinner_calculator_pb2.py generating proto file: mediapipe/calculators/core/gate_calculator_pb2.py generating proto file: mediapipe/calculators/core/split_vector_calculator_pb2.py generating proto file: mediapipe/calculators/core/packet_resampler_calculator_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/calculators/image/init.py generating proto file: mediapipe/calculators/image/opencv_image_encoder_calculator_pb2.py generating proto file: mediapipe/calculators/image/image_cropping_calculator_pb2.py generating proto file: mediapipe/calculators/image/segmentation_smoothing_calculator_pb2.py generating proto file: mediapipe/calculators/image/mask_overlay_calculator_pb2.py generating proto file: mediapipe/calculators/image/scale_image_calculator_pb2.py generating proto file: mediapipe/calculators/image/recolor_calculator_pb2.py generating proto file: mediapipe/calculators/image/image_transformation_calculator_pb2.py generating proto file: mediapipe/calculators/image/warp_affine_calculator_pb2.py generating proto file: mediapipe/calculators/image/image_clone_calculator_pb2.py generating proto file: mediapipe/calculators/image/set_alpha_calculator_pb2.py generating proto file: mediapipe/calculators/image/bilateral_filter_calculator_pb2.py generating proto file: mediapipe/calculators/image/opencv_encoded_image_to_image_frame_calculator_pb2.py generating proto file: mediapipe/calculators/image/feature_detector_calculator_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/calculators/tflite/init.py generating proto file: mediapipe/calculators/tflite/tflite_tensors_to_segmentation_calculator_pb2.py generating proto file: mediapipe/calculators/tflite/tflite_inference_calculator_pb2.py generating proto file: mediapipe/calculators/tflite/tflite_custom_op_resolver_calculator_pb2.py generating proto file: mediapipe/calculators/tflite/tflite_tensors_to_landmarks_calculator_pb2.py generating proto file: mediapipe/calculators/tflite/tflite_converter_calculator_pb2.py generating proto file: mediapipe/calculators/tflite/tflite_tensors_to_classification_calculator_pb2.py generating proto file: mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator_pb2.py generating proto file: mediapipe/calculators/tflite/ssd_anchors_calculator_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/calculators/tensor/init.py generating proto file: mediapipe/calculators/tensor/image_to_tensor_calculator_pb2.py generating proto file: mediapipe/calculators/tensor/tensor_converter_calculator_pb2.py generating proto file: mediapipe/calculators/tensor/tensors_to_detections_calculator_pb2.py generating proto file: mediapipe/calculators/tensor/tensors_to_floats_calculator_pb2.py generating proto file: mediapipe/calculators/tensor/inference_calculator_pb2.py generating proto file: mediapipe/calculators/tensor/tensors_to_segmentation_calculator_pb2.py generating proto file: mediapipe/calculators/tensor/tensors_to_landmarks_calculator_pb2.py generating proto file: mediapipe/calculators/tensor/tensors_to_classification_calculator_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/gpu/init.py generating proto file: mediapipe/gpu/gl_scaler_calculator_pb2.py generating proto file: mediapipe/gpu/gpu_origin_pb2.py generating proto file: mediapipe/gpu/copy_calculator_pb2.py generating proto file: mediapipe/gpu/scale_mode_pb2.py generating proto file: mediapipe/gpu/gl_context_options_pb2.py generating proto file: mediapipe/gpu/gl_surface_sink_calculator_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/modules/face_geometry/init.py generating proto file: mediapipe/modules/face_geometry/geometry_pipeline_calculator_pb2.py generating proto file: mediapipe/modules/face_geometry/effect_renderer_calculator_pb2.py generating proto file: mediapipe/modules/face_geometry/env_generator_calculator_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/modules/face_geometry/protos/init.py generating proto file: mediapipe/modules/face_geometry/protos/face_geometry_pb2.py generating proto file: mediapipe/modules/face_geometry/protos/geometry_pipeline_metadata_pb2.py generating proto file: mediapipe/modules/face_geometry/protos/mesh_3d_pb2.py generating proto file: mediapipe/modules/face_geometry/protos/environment_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/modules/objectron/calculators/init.py generating proto file: mediapipe/modules/objectron/calculators/lift_2d_frame_annotation_to_3d_calculator_pb2.py generating proto file: mediapipe/modules/objectron/calculators/object_pb2.py generating proto file: mediapipe/modules/objectron/calculators/filter_detection_calculator_pb2.py generating proto file: mediapipe/modules/objectron/calculators/frame_annotation_tracker_calculator_pb2.py generating proto file: mediapipe/modules/objectron/calculators/tensors_to_objects_calculator_pb2.py generating proto file: mediapipe/modules/objectron/calculators/belief_decoder_config_pb2.py generating proto file: mediapipe/modules/objectron/calculators/frame_annotation_to_rect_calculator_pb2.py generating proto file: mediapipe/modules/objectron/calculators/a_r_capture_metadata_pb2.py generating proto file: mediapipe/modules/objectron/calculators/tflite_tensors_to_objects_calculator_pb2.py generating proto file: mediapipe/modules/objectron/calculators/annotation_data_pb2.py generating proto file: mediapipe/modules/objectron/calculators/camera_parameters_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/modules/holistic_landmark/calculators/init.py generating proto file: mediapipe/modules/holistic_landmark/calculators/roi_tracking_calculator_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/util/init.py generating proto file: mediapipe/util/audio_decoder_pb2.py generating proto file: mediapipe/util/render_data_pb2.py generating proto file: mediapipe/util/color_pb2.py adding init file: /home/hungduong/mediapipe-0.8.7/mediapipe/util/tracking/init.py generating proto file: mediapipe/util/tracking/frame_selection_solution_evaluator_pb2.py generating proto file: mediapipe/util/tracking/tone_estimation_pb2.py generating proto file: mediapipe/util/tracking/flow_packager_pb2.py generating proto file: mediapipe/util/tracking/box_tracker_pb2.py generating proto file: mediapipe/util/tracking/push_pull_filtering_pb2.py generating proto file: mediapipe/util/tracking/camera_motion_pb2.py generating proto file: mediapipe/util/tracking/motion_models_pb2.py generating proto file: mediapipe/util/tracking/tone_models_pb2.py generating proto file: mediapipe/util/tracking/tracking_pb2.py generating proto file: mediapipe/util/tracking/frame_selection_pb2.py generating proto file: mediapipe/util/tracking/motion_saliency_pb2.py generating proto file: mediapipe/util/tracking/box_detector_pb2.py generating proto file: mediapipe/util/tracking/region_flow_pb2.py generating proto file: mediapipe/util/tracking/motion_analysis_pb2.py generating proto file: mediapipe/util/tracking/region_flow_computation_pb2.py generating proto file: mediapipe/util/tracking/motion_estimation_pb2.py generating proto file: mediapipe/util/tracking/tracked_detection_manager_config_pb2.py running bdist_wheel running build running build_binary_graphs generating binarypb: mediapipe/modules/hand_landmark/hand_landmark_tracking_gpu Starting local Bazel server and connecting to it... WARNING: ignoring LD_PRELOAD in environment. DEBUG: /home/hungduong/.cache/bazel/_bazel_hungduong/71c0e38a5a957b4ab7af694cb872a320/external/org_tensorflow/third_party/repo.bzl:109:14: Warning: skipping import of repository 'com_google_absl' because it already exists. DEBUG: /home/hungduong/.cache/bazel/_bazel_hungduong/71c0e38a5a957b4ab7af694cb872a320/external/org_tensorflow/third_party/repo.bzl:109:14: Warning: skipping import of repository 'com_google_protobuf' because it already exists. DEBUG: /home/hungduong/.cache/bazel/_bazel_hungduong/71c0e38a5a957b4ab7af694cb872a320/external/org_tensorflow/third_party/repo.bzl:109:14: Warning: skipping import of repository 'com_google_googletest' because it already exists. DEBUG: /home/hungduong/.cache/bazel/_bazel_hungduong/71c0e38a5a957b4ab7af694cb872a320/external/org_tensorflow/third_party/repo.bzl:109:14: Warning: skipping import of repository 'com_github_gflags_gflags' because it already exists. DEBUG: /home/hungduong/.cache/bazel/_bazel_hungduong/71c0e38a5a957b4ab7af694cb872a320/external/org_tensorflow/third_party/repo.bzl:109:14: Warning: skipping import of repository 'build_bazel_rules_apple' because it already exists. DEBUG: /home/hungduong/.cache/bazel/_bazel_hungduong/71c0e38a5a957b4ab7af694cb872a320/external/org_tensorflow/third_party/repo.bzl:109:14: Warning: skipping import of repository 'build_bazel_rules_swift' because it already exists. DEBUG: /home/hungduong/.cache/bazel/_bazel_hungduong/71c0e38a5a957b4ab7af694cb872a320/external/org_tensorflow/third_party/repo.bzl:109:14: Warning: skipping import of repository 'build_bazel_apple_support' because it already exists. DEBUG: /home/hungduong/.cache/bazel/_bazel_hungduong/71c0e38a5a957b4ab7af694cb872a320/external/org_tensorflow/third_party/repo.bzl:109:14: Warning: skipping import of repository 'bazel_skylib' because it already exists. DEBUG: /home/hungduong/.cache/bazel/_bazel_hungduong/71c0e38a5a957b4ab7af694cb872a320/external/org_tensorflow/third_party/repo.bzl:109:14: Warning: skipping import of repository 'pybind11' because it already exists. DEBUG: /home/hungduong/.cache/bazel/_bazel_hungduong/71c0e38a5a957b4ab7af694cb872a320/external/local_config_cuda/cuda/build_defs.bzl:104:14: Warning: '--define=using_cuda_nvcc' will be unsupported soon. Use '--@local_config_cuda//:enable_cuda' instead. INFO: Analyzed target //mediapipe/modules/hand_landmark:hand_landmark_tracking_gpu (132 packages loaded, 6915 targets configured). INFO: Found 1 target... ERROR: /home/hungduong/mediapipe-0.8.7/mediapipe/framework/formats/BUILD:159:11: C++ compilation of rule '//mediapipe/framework/formats:location' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -MD -MF bazel-out/aarch64-opt/bin/mediapipe/framework/formats/_objs/location/location.pic.d ... (remaining 64 argument(s) skipped) In file included from ./mediapipe/framework/formats/location.h:44, from mediapipe/framework/formats/location.cc:15: ./mediapipe/framework/port/opencv_core_inc.h:18:10: fatal error: opencv2/core/version.hpp: No such file or directory
include <opencv2/core/version.hpp>
^
~~~~~compilation terminated. Target //mediapipe/modules/hand_landmark:hand_landmark_tracking_gpu failed to build Use --verbose_failures to see the command lines of failed build steps. INFO: Elapsed time: 41.323s, Critical Path: 8.05s INFO: 14 processes: 14 internal. FAILED: Build did NOT complete successfully
I'm building mediapipe for Raspberry Pi 4 on RPi4 itself using your guide.
Importing mediapipe in python3 fails due to the following error:
If I disable lapack lib in third_parties/BUILD by
"WITH_LAPACK": "OFF"
then the import works fine.I'm concerned about the latency of matrix calculations without lapack, though I do not have data to support it. However what I've got with CPU is about 5 fps, what is 2-3 times slower than what I see in comments on mediapipe repo.
Please any thoughts to guide me. Thanks.