How to go about integrating mediapipe's hand tracking into an existing Visual Studio C++ project?

[CristianNCC]

I must admit that I am quite a newb when it comes to build systems. So far I've managed to build the hello_word and the hand_tracking examples on Windows (going through quite a few hoops along the way). Of course, building those examples gives me an executable, which I can't integrate in another project.

What I'd like to do is have mediapipe as a dll or something similar so I can pass the framework an image and get joint coordinates in return, ideally. Can someone point me in the right direction?

[BitLoose]

An example of this would be useful as I can't see any clear explanation of how mediapipe is integrated into a standard c++ application.

[ttamas0713]

An example of this would be useful as I can't see any clear explanation of how mediapipe is integrated into a standard c++ application.

You can copy or modify the existing file called _demo_run_graphmain.cc or _demo_run_graph_maingpu.cc at /mediapipe/examples/desktop. Also, _simple_maingraph.cc (c++ file of hello_world example) can be useful to learn how to start a graph and get it's output.

[CristianNCC]

An example of this would be useful as I can't see any clear explanation of how mediapipe is integrated into a standard c++ application.

You can copy or modify the existing file called _demo_run_graphmain.cc or _demo_run_graph_maingpu.cc at /mediapipe/examples/desktop. Also, _simple_maingraph.cc (c++ file of hello_world example) can be useful to learn how to start a graph and get it's output.

Well, I've tried that in many ways already :/. What I mean is I've tried to change the Bazel BUILD file in _/mediapipe/examples/desktop for demo_run_graph_main so it builds a DLL that you can simply plug into another application along with a header file. From what I've seen, the cc_library Bazel rules offer you a "linkstatic" flag that is true by default and it can be set to false. The output binaries didn't seem to differ when I changed that one flag though...

[jerrysheen]

I'm also interested in this, have u guys already figure out how to do this?

[shubhamvad]

Hi guys, any update on this ? I am also interested to know the solution on this. Thanks in advance.

[ghost]

Even I'm trying to figure out the same, eagerly looking forward to this feature

[FabianWildgrube]

Same here. I would be very interested to see how this could be done.

[vamsee-asu2019]

Have you guys figured this out ? I am trying to use the mediapipe in another c++ project. Any leads would be helpful.

[travisjayday]

Bump. As a fellow noob to the framework, the task of integration into existing C++ projects seems really daunting without guidance.

[BitLoose]

I agree, a lot of serious developers will be put off until there is at least a working example of how to interface with C==++.

[carter54]

Same here. An example of how to integrating mediapipe application into an existing c++ project will be very helpful~

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler[bot]]

Closing as stale. Please reopen if you'd like to work on this further.

[BitLoose]

Any update on a c++ sample for MediaPipe?

MediaPipe is overlooking a lot of professional users and projects for not having c++ api examples, how long would it take someone in the MP team to do this, half an hour?

Best regards Rob Colclough CTO Sentel Advance Systems - ViewPoint Group T. US +1 215 240 6030 T. UK +44 (0)20 8144 3150 (Low-rate 03333 442 120) www.Sentel-Advance.com Skype: ViewPoint3D YouTube: www.youtube.com/viewpoint3d

Viva3D software offers a fresh and immediate way to create stunning 3D presentations, info TV, content and simulations with real-time data for almost any type of application.

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

On 10/08/2021 09:13, GowrojiSunil wrote:

Reopened #1162 https://github.com/google/mediapipe/issues/1162.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/google/mediapipe/issues/1162#event-5134014872, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADYZGDZMWIWYIY4PYSFSFM3T4DNT7ANCNFSM4R5XRAXQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.

[hadon]

Due to limited resources, MediaPipe team is not actively working on this.

I see some discussion about consuming a bazel build artifacts from cmake at: https://groups.google.com/g/bazel-discuss/c/LP9tyCws_cg?pli=1. It looks like a custom cmake rule (add_custom_target) can be used to build and consume a bazel target, such as the mediapipe "calculator_framework". One difficulty has been grouping the build artifacts into a single object file. It looks like bazel team is addressing this difficulty in https://github.com/bazelbuild/bazel/issues/1920.

One additional difficulty is that, for now, the MediaPipe C++ library only works within a single shared library or DLL, due to global "registry" singletons. For now, you will need to call the MediaPipe APIs only from within a single DLL, which can in turn export its own public functions.

The related issue google/mediapipe#1746 provides a hint about building a DLL from bazel, https://stackoverflow.com/questions/49664449/can-bazel-build-dll.

[BitLoose]

That's crazy, c++ is the only open standards OO language used in IOT and other embedded devices, it is the most flexible and efficient language available, reports say its popularity is in the top 3 languages and rising, yet MediaPipe doesn't support it.

IMHO you need to assign resources to this issue or your project will be handicapped.

Best regards Rob Colclough CTO Sentel Advance Systems - ViewPoint Group T. US +1 215 240 6030 T. UK +44 (0)20 8144 3150 (Low-rate 03333 442 120) www.Sentel-Advance.com Skype: ViewPoint3D YouTube: www.youtube.com/viewpoint3d

Viva3D software offers a fresh and immediate way to create stunning 3D presentations, info TV, content and simulations with real-time data for almost any type of application.

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

On 11/08/2021 23:13, Hadon Nash wrote:

Due to limited resources, MediaPipe team is not actively working on this.

I see some discussion about consuming a bazel build artifacts from cmake at: https://groups.google.com/g/bazel-discuss/c/LP9tyCws_cg?pli=1 https://groups.google.com/g/bazel-discuss/c/LP9tyCws_cg?pli=1. It looks like a custom cmake rule (add_custom_target) can be used to build a bazel target, such as the mediapipe calculator_framework. One difficulty has been grouping the build artifacts into a single object file. It looks like bazel team is addressing this difficulty in bazelbuild/bazel#1920 https://github.com/bazelbuild/bazel/issues/1920.

One additional difficulty is that, for now, the MediaPipe C++ library only works within a single shared library or DLL, due to global "registry" singletons. For now, you will need to call the MediaPipe APIs only from within a single DLL, which can in turn export its own public functions.

The related issue #1746 https://github.com/google/mediapipe/issues/1746 provides a hint about building a DLL from bazel, https://stackoverflow.com/questions/49664449/can-bazel-build-dll https://stackoverflow.com/questions/49664449/can-bazel-build-dll.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/google/mediapipe/issues/1162#issuecomment-897196714, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADYZGD7PNNSWYOT2IE463LLT4LY2DANCNFSM4R5XRAXQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[willstott101]

This is still very much a real issue affecting a lot of people. I'm not sure closing it is helpful - even though the mediapipe team doesn't have the resources to approach it right now.

[BitLoose]

If anyone interested in doing this work pls email me, @.***

On 19/08/2021 11:27, Will Stott wrote:

This is still very much a real issue affecting a lot of people. I'm not sure closing it is helpful - even though the mediapipe team doesn't have the resources to approach it right now.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/google/mediapipe/issues/1162#issuecomment-901798180, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADYZGD7NJITCGTFJRX7ETVDT5TL75ANCNFSM4R5XRAXQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.

[mahtaabdn]

any update? I have to use mediapipe in a C++ project, for my master thesis. Any leads would be helpful.

[FabianWildgrube]

any update? I have to use mediapipe in a C++ project, for my master thesis. Any leads would be helpful.

If your project is a standalone and does not need to be integrated into an existing C++ project that uses MSVC (and thus Visual Studio), you could use a dockerized setup to still run it on windows. I did that in this project: https://github.com/FabianWildgrube/PupilSizeTracker. Maybe that helps.

As for integration with a Visual C++ project I've had no luck, though.

[mahtaabdn]

I have to integrate mediapipe to an existing C++ project in Qt.

[willstott101]

You basically need to add a target like this one: https://github.com/willstott101/meta-mediapipe/blob/master/recipes-framework/mediapipe/files/0002-Build-library.patch then copy all the headers from bazel's working directories and try linking to it.

If you're already linking to any of the libs that mediapipe uses you may have trouble - you'll need to make mediapipe link to those which is a bit of a fiddle - but doable

FYI this is for a dynamic lib using GPU on linux - I haven't tried on windows you may only need to replace .so with .dll

[maletsden]

Hi, check my PR with an example of creating a FaceMesh Windows DLL example - Face mesh Windows DLL desktop example. This example generates standalone .dll that, that I further use in my other Visual Studio Project, by simply loading DLL at runtime from C++ (on Windows - Run-Time Dynamic Linking)

[fortminors]

@maletsden Hello! Thanks for providing that info. How would I build the .dll? I am new to Bazel, and tried to run bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/face_mesh_dll:face_mesh_lib but it gave me this output with an error:

DEBUG: Rule 'rules_foreign_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "c2cdcf55ffaf49366725639e45dedd449b8c3fe22b54e31625eb80ce3a240f1e"
DEBUG: Repository rules_foreign_cc instantiated at:
  C:/users/fortm/desktop/mediapipe-face_mesh_dll_example/WORKSPACE:42:13: in <toplevel>
Repository rule http_archive defined at:
  C:/users/fortm/_bazel_fortm/hhj5wzn5/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
WARNING: Download from http://mirror.tensorflow.org/github.com/bazelbuild/rules_closure/archive/cf1e44edb908e9616030cc83d085989b8e6cd6df.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
WARNING: Download from http://mirror.tensorflow.org/github.com/tensorflow/runtime/archive/44c890cd46fd9aad2b3bf4b13c5f36d8cff795c3.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
DEBUG: C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'com_google_absl' because it already exists.
DEBUG: C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'com_google_protobuf' because it already exists.
DEBUG: C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'com_google_googletest' because it already exists.
DEBUG: C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'com_github_gflags_gflags' because it already exists.
DEBUG: C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'build_bazel_rules_apple' because it already exists.
DEBUG: C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'build_bazel_rules_swift' because it already exists.
DEBUG: C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'build_bazel_apple_support' because it already exists.
DEBUG: C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'bazel_skylib' because it already exists.
DEBUG: C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/repo.bzl:108:14:
Warning: skipping import of repository 'pybind11' because it already exists.
DEBUG: C:/users/fortm/_bazel_fortm/hhj5wzn5/external/tf_runtime/third_party/cuda/dependencies.bzl:51:10: The following command will download NVIDIA proprietary software. By using the software you agree to comply with the terms of the license agreement that accompanies the software. If you do not agree to the terms of the license agreement, do not use the software.
DEBUG: Rule 'rules_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "86bf1c24e8f859fb5c66d931f33dc8283427e7370d2b0d178658893bcd3c49e9"
DEBUG: Repository rules_cc instantiated at:
  C:/users/fortm/desktop/mediapipe-face_mesh_dll_example/WORKSPACE:36:13: in <toplevel>
Repository rule http_archive defined at:
  C:/users/fortm/_bazel_fortm/hhj5wzn5/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
INFO: Build option --compilation_mode has changed, discarding analysis cache.
INFO: Repository local_execution_config_python instantiated at:
  C:/users/fortm/desktop/mediapipe-face_mesh_dll_example/WORKSPACE:408:14: in <toplevel>
  C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/tensorflow/workspace2.bzl:1087:19: in workspace
  C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/tensorflow/workspace2.bzl:85:27: in _tf_toolchains
  C:/users/fortm/_bazel_fortm/hhj5wzn5/external/tf_toolchains/toolchains/remote_config/configs.bzl:6:28: in initialize_rbe_configs
  C:/users/fortm/_bazel_fortm/hhj5wzn5/external/tf_toolchains/toolchains/remote_config/rbe_config.bzl:158:27: in _tensorflow_local_config
Repository rule local_python_configure defined at:
  C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/py/python_configure.bzl:275:41: in <toplevel>
ERROR: An error occurred during the fetch of repository 'local_execution_config_python':
   Traceback (most recent call last):
        File "C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/py/python_configure.bzl", line 209, column 22, in _create_local_python_repository
                _check_python_bin(repository_ctx, python_bin)
        File "C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/py/python_configure.bzl", line 143, column 52, in _check_python_bin
                result = raw_exec(repository_ctx, [get_bash_bin(repository_ctx), "-c", cmd])
        File "C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/remote_config/common.bzl", line 88, column 26, in get_bash_bin
                bash_bin_path = which(repository_ctx, "bash")
        File "C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/remote_config/common.bzl", line 27, column 22, in which
                out = execute(
        File "C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/remote_config/common.bzl", line 230, column 13, in execute
                fail(
Error in fail: Repository command failed
INFO: Could not find files for the given pattern(s).
ERROR: Error fetching repository: Traceback (most recent call last):
        File "C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/py/python_configure.bzl", line 209, column 22, in _create_local_python_repository
                _check_python_bin(repository_ctx, python_bin)
        File "C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/py/python_configure.bzl", line 143, column 52, in _check_python_bin
                result = raw_exec(repository_ctx, [get_bash_bin(repository_ctx), "-c", cmd])
        File "C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/remote_config/common.bzl", line 88, column 26, in get_bash_bin
                bash_bin_path = which(repository_ctx, "bash")
        File "C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/remote_config/common.bzl", line 27, column 22, in which
                out = execute(
        File "C:/users/fortm/_bazel_fortm/hhj5wzn5/external/org_tensorflow/third_party/remote_config/common.bzl", line 230, column 13, in execute
                fail(
Error in fail: Repository command failed
INFO: Could not find files for the given pattern(s).
ERROR: Analysis of target '//mediapipe/examples/desktop/face_mesh_dll:face_mesh_lib' failed; build aborted: Repository command failed
INFO: Could not find files for the given pattern(s).
INFO: Elapsed time: 0.289s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 11 targets configured)

How did you manage to build the DLL? Thanks!

UPDATE

I fixed it by following this issue and by installing OpenCV 3.4.10 at C:/opencv and numpy through pip install numpy. Also, you have to have python 3.6 installed and set the PYTHON_BIN_PATH with:

set PYTHON_BIN_PATH=C:/Users/<username>/AppData/Local/Programs/Python/Python36/python.exe

[fortminors]

@maletsden Since I've managed to build successfully, I can't find the dll file that it should produce. I was searching for face_mesh_lib.dll. I fail to understand where Bazel would put the files. I would appreciate any help.

[mahtaabdn]

https://github.com/google/mediapipe/issues/2586#issue-1005490666

[BaiL939]

Thanks for the positive discussion, but mediapipe is not very friendly to C++.But my ability is really insufficient, now the C++ development environment is not built well.But after reading everyone's discussion, I still feel that there is no concrete and practical answer

[424778940z]

Please consider move to CMake or at least provided a way to build full library and headers in Bazel. The real problem here is Bazel is really limited both on function and documents.

It is the end of 2021, and Bazel as a "build tool" is not able to produce a static library natively is ridiculous. To make the things worse the official devs says they have NO plan for it at all. And BTW the issue it's self already FIVE YEARS old.

Also for Bazel, there is NO WAY to install related headers to a folder, it may not be an issue for "package based" languages. But C++ need those headers to use the output library.

And speaking of the document, it is a huge mess. Some of the pages are ok, some are basiclly empty such as this one https://docs.bazel.build/versions/main/skylark/lib/file_provider.html

Extensions > API reference > file_provider
file_provider
An interface for rules that provide files.

And the documents are only for basic language wise, there is NO ducment for most of the rules, espicially on how to interact with them. Even something as common as cc_library, there are document on how to use them. But there is nothing descipt how to get CCInfo (that contains include and link info) from it, you never find out without dig through source.

[Jerryzhangzhao]

You can consider convert the mediapipe .tflite model to onnx or even tensorRT model, and then do the inference with in c++ project.

[fortminors]

@Jerryzhangzhao but for example the face detector model outputs anchors. How do you convert them to bounding rects? I haven't figured it out yet. So there is a post-processing problem

[Jerryzhangzhao]

@Jerryzhangzhao but for example the face detector model outputs anchors. How do you convert them to bounding rects? I haven't figured it out yet. So there is a post-processing problem

Yes, the post-processing step is important. First of all, from the paper blazeFace , we know that there are two type 896 anchors: with the visualization of the tflite model, the output including classificators [1x896x1] and regressors [1x896x16] With a sigmoid activation applied on classificators would generate the confidence score of each boundingbox, and teh first four element in each 16 output of regressors is [y_offset, x_offset, height, width] of the boudningbox. The above information can be found in the source code of mediapipe tflite_tensors_to_detections_calculator.cc where the code segment： line 326:

// Filter classes by scores.
    for (int i = 0; i < num_boxes_; ++i) {
      int class_id = -1;
      float max_score = -std::numeric_limits<float>::max();
      // Find the top score for box i.
      for (int score_idx = 0; score_idx < num_classes_; ++score_idx) {
        if (ignore_classes_.find(score_idx) == ignore_classes_.end()) {
          auto score = raw_scores[i * num_classes_ + score_idx];
          if (options_.sigmoid_score()) {
            if (options_.has_score_clipping_thresh()) {
              score = score < -options_.score_clipping_thresh()
                          ? -options_.score_clipping_thresh()
                          : score;
              score = score > options_.score_clipping_thresh()
                          ? options_.score_clipping_thresh()
                          : score;
            }
            score = 1.0f / (1.0f + std::exp(-score));
          }
          if (max_score < score) {
            max_score = score;
            class_id = score_idx;
          }
        }
      }
      detection_scores[i] = max_score;
      detection_classes[i] = class_id;
    }

line 94:

void ConvertAnchorsToRawValues(const std::vector<Anchor>& anchors,
                               int num_boxes, float* raw_anchors) {
  CHECK_EQ(anchors.size(), num_boxes);
  int box = 0;
  for (const auto& anchor : anchors) {
    raw_anchors[box * kNumCoordsPerBox + 0] = anchor.y_center();
    raw_anchors[box * kNumCoordsPerBox + 1] = anchor.x_center();
    raw_anchors[box * kNumCoordsPerBox + 2] = anchor.h();
    raw_anchors[box * kNumCoordsPerBox + 3] = anchor.w();
    ++box;
  }
}

line 590：

 float y_center = raw_boxes[box_offset];
    float x_center = raw_boxes[box_offset + 1];
    float h = raw_boxes[box_offset + 2];
    float w = raw_boxes[box_offset + 3];
    if (options_.reverse_output_order()) {
      x_center = raw_boxes[box_offset];
      y_center = raw_boxes[box_offset + 1];
      w = raw_boxes[box_offset + 2];
      h = raw_boxes[box_offset + 3];
    }

[fortminors]

@Jerryzhangzhao Thanks a lot for sharing this! It's very useful

[pkoppise]

You basically need to add a target like this one: https://github.com/willstott101/meta-mediapipe/blob/master/recipes-framework/mediapipe/files/0002-Build-library.patch then copy all the headers from bazel's working directories and try linking to it.

If you're already linking to any of the libs that mediapipe uses you may have trouble - you'll need to make mediapipe link to those which is a bit of a fiddle - but doable

FYI this is for a dynamic lib using GPU on linux - I haven't tried on windows you may only need to replace .so with .dll

HI,

I have integrated the mediapipe as c++ (hand tracking) qt project for webos (assuming CPU) , however the screen is not rendering with latest frames after polling , it's getting returned

size_t frame_timestamp_us = static_cast<double>(cv::getTickCount()) / static_cast<double>(cv::getTickFrequency()) * 1e6;

absl::Status pstatus = graph.AddPacketToInputStream("input_video", mediapipe::Adopt(input_frame.release())
                      .At(mediapipe::Timestamp(frame_timestamp_us)));

// Get the graph result packet, or stop if that fails.
mediapipe::Packet packet;
if (!poller.Next(&packet)) return;
auto& output_frame = packet.Get<mediapipe::ImageFrame>();

Could you please help if you faced similar problem on webos or any other platform

[asprecic]

You basically need to add a target like this one: https://github.com/willstott101/meta-mediapipe/blob/master/recipes-framework/mediapipe/files/0002-Build-library.patch then copy all the headers from bazel's working directories and try linking to it. If you're already linking to any of the libs that mediapipe uses you may have trouble - you'll need to make mediapipe link to those which is a bit of a fiddle - but doable FYI this is for a dynamic lib using GPU on linux - I haven't tried on windows you may only need to replace .so with .dll

HI,

I have integrated the mediapipe as c++ (hand tracking) qt project for webos (assuming CPU) , however the screen is not rendering with latest frames after polling , it's getting returned

size_t frame_timestamp_us = static_cast<double>(cv::getTickCount()) / static_cast<double>(cv::getTickFrequency()) * 1e6;

absl::Status pstatus = graph.AddPacketToInputStream("input_video", mediapipe::Adopt(input_frame.release())
                      .At(mediapipe::Timestamp(frame_timestamp_us)));

// Get the graph result packet, or stop if that fails.
mediapipe::Packet packet;
if (!poller.Next(&packet)) return;
auto& output_frame = packet.Get<mediapipe::ImageFrame>();

Could you please help if you faced similar problem on webos or any other platform

I also had that problem where poller would return. It is required that the tflite model is located at "mediapipe/modules/selfie_segmentation/selfie_segmentation.tflite" of wherever your binary is. This path can be changed if you dig into mediapipe.

[asprecic]

You basically need to add a target like this one: https://github.com/willstott101/meta-mediapipe/blob/master/recipes-framework/mediapipe/files/0002-Build-library.patch then copy all the headers from bazel's working directories and try linking to it.

If you're already linking to any of the libs that mediapipe uses you may have trouble - you'll need to make mediapipe link to those which is a bit of a fiddle - but doable

FYI this is for a dynamic lib using GPU on linux - I haven't tried on windows you may only need to replace .so with .dll

Thank you very much for this help. This was the missing puzzle I needed to produce .so and be able to interface to mediapipe from other projects.

I was able to produce .so on linux, and link to it from another Qt project. I did have to manually copy the header file for interface which I created.

Overall, it is possible to integrate this into existing c++ projects, it's just not easy, simple nor well documented. You're pretty much on your own to make it happen. I had to dig through a lot of mediapipe and make changes that get bazel to build the .so how I want it (stripped of useless unused junk libraries).

[pkoppise]

@asprecic @willstott101 Now able to run hand-tracking cpu example as an QML application. a) Generated libmediapipe.so b) Resolved all the dependencies (absl) c) Created a QML application (backend c++, where i am using mediapipe, opencv) and in .pro file linked all needed dependencies Thanks a lot for the help

[asprecic]

Hey everyone,

I made an example Qt application to demonstrate how to integrate Mediapipe as a dynamic library in Linux. It runs many examples from Mediapipe examples, including the hand tracking.

You can find the example application here: https://github.com/asprecic/mediapipe-qt-integration-example

and the corresponding changes in Mediapipe in this fork here: https://github.com/asprecic/mediapipe

[LiuPeiqiCN]

Hey everyone,

I made an example Qt application to demonstrate how to integrate Mediapipe as a dynamic library in Linux. It runs many examples from Mediapipe examples, including the hand tracking.

You can find the example application here: https://github.com/asprecic/mediapipe-qt-integration-example

and the corresponding changes in Mediapipe in this fork here: https://github.com/asprecic/mediapipe

I try to build mediapipe in windows, build is successful, but when I run demo, I got an error:

[libprotobuf ERROR external/com_google_protobuf/src/google/protobuf/text_format.cc:309] Error parsing text-format 
mediapipe.CalculatorGraphConfig: 33:100: Could not find type 
"type.googleapis.com/mediapipe.DetectionsToRenderDataCalculatorOptions" stored in google.protobuf.Any.
F20211205 10:07:29.508597 24436 parse_text_proto.h:32] Check failed: ParseTextProto(input, &result)
*** Check failure stack trace: ***

I found this issue: https://github.com/google/mediapipe/issues/1953, but it did not work.

[problemSolvingProgramming]

@CristianNCC ,@ttamas0713 Excuse me, I'm having your problem. My goal is to be able to use the mediapipe pose tracking program in another C ++ project in Visal Studio 2019. Can you help me how to do this? https://github.com/google/mediapipe/tree/master/mediapipe/examples/desktop/pose_tracking

Or put the code in a repository.

[LiuPeiqiCN]

I can call mediapipe's python API in my C++ program, but the downside is that I have to integrate a python embed, and python API is limited in functionality compared to C++ API.

[BitLoose]

I agree, its less than optimal, I'm waiting until I can test this in Visual Studio C++ without all the complexities?inefficiency.

contents of this information is strictly prohibited.

On 08/03/2022 11:10, Peiqi.Liu wrote:

I can call mediapipe's python API in my C++ program, but the downside is that I have to integrate a |python embed|, and python API is limited in functionality compared to C++ API.

— Reply to this email directly, view it on GitHub https://github.com/google/mediapipe/issues/1162#issuecomment-1061664443, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADYZGDZO372FOTODF36FHU3U64YTVANCNFSM4R5XRAXQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

[vishal-ahirwar]

@asprecic @willstott101 Now able to run hand-tracking cpu example as an QML application. a) Generated libmediapipe.so b) Resolved all the dependencies (absl) c) Created a QML application (backend c++, where i am using mediapipe, opencv) and in .pro file linked all needed dependencies Thanks a lot for the help Can you please tell me how you did that??????????//// Your whole procedure like how you build dynamic library ???????/// Thanks in Advanced.

[Mario-td]

What I did is building the mediapipe C++ program using bazel, then execute it from a different process and communicate through shared memory. Here is the demo: https://github.com/Mario-td/Simplified-hand-tracking-with-Mediapipe-CPP

[problemSolvingProgramming]

@Mario-td Thank you, please do the same for pose tracking .

On Sat, Apr 23, 2022 at 9:06 AM Mario-td @.***> wrote:

What I did is building the mediapipe C++ program using bazel, then execute it from a different process and communicate through shared memory. Here is the demo: https://github.com/Mario-td/Simplified-hand-tracking-with-Mediapipe-CPP

— Reply to this email directly, view it on GitHub https://github.com/google/mediapipe/issues/1162#issuecomment-1107527036, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2ULSVYS6VN7JHVTQM3WKDVGQNYLANCNFSM4R5XRAXQ . You are receiving this because you commented.Message ID: @.***>

[xfj81525]

i have successfully compiled a dynamic lib of face landmark detection and used it in visual studio 2019 project.

[LiuPeiqiCN]

i have successfully compiled a dynamic lib of face landmark detection and used it in visual studio 2019 project.

I can use bazel build a dynamic ".dll", but I don't know how to get a ".lib" to link it. Can you share your experience of using VS2019?

[BitLoose]

that's interesting, where can I download? Thxs

[andrechen]

i put a visual studio 2019 solution file with few selected example project files here. And If you build it (solution), they will run/debug.

all in this fork: https://github.com/andrechen/mediapipe

[lilinxiong]

i put a visual studio 2019 solution file with few selected example project files here. And If you build it (solution), they will run/debug.

all in this fork: https://github.com/andrechen/mediapipe

That's amazing. I changed some things based on bazel before, so that it can build 32-bit library, so I have a question, can this vs project build 32-bit library?