homuler / MediaPipeUnityPlugin

Unity plugin to run MediaPipe
MIT License
1.81k stars 467 forks source link

Building on Android TV. missing DLL mediapipe_jni #1268

Open DMLV-dev opened 6 days ago

DMLV-dev commented 6 days ago

Plugin Version or Commit ID

v0.15.0

Unity Version

2022.3.28f1

Your Host OS

Windows 10

Target Platform

Android

[Windows Only] Visual Studio C++ and Windows SDK Version

No response

[Linux Only] GCC/G++ and GLIBC Version

No response

[Android Only] Android Build Tools and NDK Version

Android 9

[iOS Only] XCode Version

No response

Command Sequences

I just played the sample scene Pose land make detection

Log

Screenshot_2024-10-29_222201

Additional Context

I'm building for an android TV its armv7 and I did all the setup requirements like streaming assets and there is also a mediapipe_android . It is working in my Tablet but its not working on an android box tv. can you help me with this?

image

homuler commented 4 days ago

Due to build constraints, the distributed package does not include code for armv7, so you will need to build it yourself. ref. https://github.com/homuler/MediaPipeUnityPlugin/releases/tag/v0.14.3

If you need a fat aar, either build individually for each platform or exclude the AudioClassifier from the build (e.g. python build.py build --android arm64 --android_fat_apk_cpu=armeabi-v7a,arm64-v8a --solutions=face_mesh).

Please try using the GitHub Actions workflow for building the library. cf. https://github.com/homuler/MediaPipeUnityPlugin?tab=readme-ov-file#hammer_and_wrench-installation

DMLV-dev commented 2 days ago

Sorry I'm just a beginner. can you help me step by step on how to build for armv7? do I use Docker linux for windows? or android studio?

DMLV-dev commented 2 days ago

I tried doing this with Docker linux for windows.

`E:\MediaPipeUnityPlugin-all-stripped>docker run --cpus=6 --memory=16g ^ More? --mount type=bind,src=E:\MediaPipeUnityPlugin-all-stripped/Packages,dst=/home/mediapipe/Packages ^ More? --mount type=bind,src=E:\MediaPipeUnityPlugin-all-stripped/Assets,dst=/home/mediapipe/Assets ^ More? -it mediapipe_unity:linux mediapipe@66f48448d3a8:~$ build.py build --android arm64 --android_fat_apk_cpu=armeabi-v7a,arm64-v8a --solutions=face_mesh bash: build.py: command not found mediapipe@66f48448d3a8:~$ python build.py build --android arm64 --android_fat_apk_cpu=armeabi-v7a,arm64-v8a --solutions=face_mesh INFO (build.py): Building protobuf sources... 2024/11/02 13:31:07 Downloading https://releases.bazel.build/6.1.1/release/bazel-6.1.1-linux-x86_64... Downloading: 50 MB out of 50 MB (100%) Extracting Bazel installation... Starting local Bazel server and connecting to it... WARNING: Download from http://mirror.tensorflow.org/github.com/bazelbuild/rules_closure/archive/cf1e44edb908e9616030cc83d085989b8e6cd6df.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/dcffa4094e13d56a30b2cdcdb709ce5d71b38953.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/1c154bd755153b5c6ada4bbed58facf23f6abffc.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found DEBUG: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/org_tensorflow/third_party/repo.bzl:132:14: Warning: skipping import of repository 'com_google_absl' because it already exists. DEBUG: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/org_tensorflow/third_party/repo.bzl:132:14: Warning: skipping import of repository 'com_google_benchmark' because it already exists. DEBUG: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/org_tensorflow/third_party/repo.bzl:132:14: Warning: skipping import of repository 'flatbuffers' because it already exists. DEBUG: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/org_tensorflow/third_party/repo.bzl:132:14: Warning: skipping import of repository 'com_googlesource_code_re2' because it already exists. DEBUG: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/org_tensorflow/third_party/repo.bzl:132:14: Warning: skipping import of repository 'com_google_protobuf' because it already exists. DEBUG: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/org_tensorflow/third_party/repo.bzl:132:14: Warning: skipping import of repository 'com_google_googletest' because it already exists. DEBUG: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/org_tensorflow/third_party/repo.bzl:132:14: Warning: skipping import of repository 'com_github_gflags_gflags' because it already exists. DEBUG: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/org_tensorflow/third_party/repo.bzl:132:14: Warning: skipping import of repository 'zlib' because it already exists. DEBUG: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/org_tensorflow/third_party/repo.bzl:132:14: Warning: skipping import of repository 'build_bazel_rules_apple' because it already exists. DEBUG: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/org_tensorflow/third_party/repo.bzl:132:14: Warning: skipping import of repository 'build_bazel_rules_swift' because it already exists. DEBUG: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/org_tensorflow/third_party/repo.bzl:132:14: Warning: skipping import of repository 'build_bazel_apple_support' because it already exists. DEBUG: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/org_tensorflow/third_party/repo.bzl:132:14: Warning: skipping import of repository 'xctestrunner' because it already exists. DEBUG: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/org_tensorflow/third_party/repo.bzl:132:14: Warning: skipping import of repository 'com_github_glog_glog' because it already exists. DEBUG: Rule 'rules_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "691a29db9c336349e48e04c5c2f4873f2890af5cbfa6e51f4de87fefe6169294" DEBUG: Repository rules_cc instantiated at: /home/mediapipe/WORKSPACE:66:13: in Repository rule http_archive defined at: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/bazel_tools/tools/build_defs/repo/http.bzl:372:31: in INFO: Analyzed target //mediapipe_api:mediapipe_proto_srcs (122 packages loaded, 1122 targets configured). INFO: Found 1 target... Target //mediapipe_api:mediapipe_proto_srcs up-to-date: bazel-bin/mediapipe_api/mediapipe_proto_srcs.zip INFO: Elapsed time: 166.451s, Critical Path: 44.43s INFO: 352 processes: 8 internal, 344 linux-sandbox. INFO: Build completed successfully, 352 total actions INFO (build.py): Built protobuf sources INFO (build.py): Downloading dlls... Feeds used: https://api.nuget.org/v3/index.json

Restoring NuGet package Google.Protobuf.3.20.0. GET https://api.nuget.org/v3-flatcontainer/google.protobuf/3.20.0/google.protobuf.3.20.0.nupkg OK https://api.nuget.org/v3-flatcontainer/google.protobuf/3.20.0/google.protobuf.3.20.0.nupkg 10ms Installed Google.Protobuf 3.20.0 from https://api.nuget.org/v3/index.json to /home/mediapipe/.nuget/packages/google.protobuf/3.20.0 with content hash dZkpScgy6Lg6mjj+zqQkAZR49Jyoag5zloUw/gnQZvClbAW5lKPkXOoIRoM+3vwS9UNNBDwqiAJ+2KW2ZXOkCQ==. Adding package 'Google.Protobuf.3.20.0' to folder '/home/mediapipe/.nuget' Added package 'Google.Protobuf.3.20.0' to folder '/home/mediapipe/.nuget' Restoring NuGet package System.Buffers.4.4.0. GET https://api.nuget.org/v3-flatcontainer/system.buffers/4.4.0/system.buffers.4.4.0.nupkg OK https://api.nuget.org/v3-flatcontainer/system.buffers/4.4.0/system.buffers.4.4.0.nupkg 9ms Installed System.Buffers 4.4.0 from https://api.nuget.org/v3/index.json to /home/mediapipe/.nuget/packages/system.buffers/4.4.0 with content hash AwarXzzoDwX6BgrhjoJsk6tUezZEozOT5Y9QKF94Gl4JK91I4PIIBkBco9068Y9/Dra8Dkbie99kXB8+1BaYKw==. Adding package 'System.Buffers.4.4.0' to folder '/home/mediapipe/.nuget' Added package 'System.Buffers.4.4.0' to folder '/home/mediapipe/.nuget' Restoring NuGet package System.Memory.4.5.3. GET https://api.nuget.org/v3-flatcontainer/system.memory/4.5.3/system.memory.4.5.3.nupkg OK https://api.nuget.org/v3-flatcontainer/system.memory/4.5.3/system.memory.4.5.3.nupkg 9ms Installed System.Memory 4.5.3 from https://api.nuget.org/v3/index.json to /home/mediapipe/.nuget/packages/system.memory/4.5.3 with content hash 3oDzvc/zzetpTKWMShs1AADwZjQ/36HnsufHRPcOjyRAAMLDlu2iD33MBI2opxnezcVUtXyqDXXjoFMOU9c7SA==. Adding package 'System.Memory.4.5.3' to folder '/home/mediapipe/.nuget' Added package 'System.Memory.4.5.3' to folder '/home/mediapipe/.nuget' Restoring NuGet package System.Runtime.CompilerServices.Unsafe.4.5.2. GET https://api.nuget.org/v3-flatcontainer/system.runtime.compilerservices.unsafe/4.5.2/system.runtime.compilerservices.unsafe.4.5.2.nupkg OK https://api.nuget.org/v3-flatcontainer/system.runtime.compilerservices.unsafe/4.5.2/system.runtime.compilerservices.unsafe.4.5.2.nupkg 9ms Installed System.Runtime.CompilerServices.Unsafe 4.5.2 from https://api.nuget.org/v3/index.json to /home/mediapipe/.nuget/packages/system.runtime.compilerservices.unsafe/4.5.2 with content hash wprSFgext8cwqymChhrBLu62LMg/1u92bU+VOwyfBimSPVFXtsNqEWC92Pf9ofzJFlk4IHmJA75EDJn1b2goAQ==. Adding package 'System.Runtime.CompilerServices.Unsafe.4.5.2' to folder '/home/mediapipe/.nuget' Added package 'System.Runtime.CompilerServices.Unsafe.4.5.2' to folder '/home/mediapipe/.nuget' Restoring NuGet package Microsoft.Unity.Analyzers.1.11.1. GET https://api.nuget.org/v3-flatcontainer/microsoft.unity.analyzers/1.11.1/microsoft.unity.analyzers.1.11.1.nupkg OK https://api.nuget.org/v3-flatcontainer/microsoft.unity.analyzers/1.11.1/microsoft.unity.analyzers.1.11.1.nupkg 9ms Installed Microsoft.Unity.Analyzers 1.11.1 from https://api.nuget.org/v3/index.json to /home/mediapipe/.nuget/packages/microsoft.unity.analyzers/1.11.1 with content hash +Xy7OwNmokwvduMHZankM7KqNkGQwyQnd+FgIEsJ+gWKNVJxC6nF6FiOYSc1WgBpgQr8Bt5RBZ7agk8e6nGKpQ==. Adding package 'Microsoft.Unity.Analyzers.1.11.1' to folder '/home/mediapipe/.nuget' Added package 'Microsoft.Unity.Analyzers.1.11.1' to folder '/home/mediapipe/.nuget' Restoring NuGet package ErrorProne.NET.CoreAnalyzers.0.4.0-beta.1. GET https://api.nuget.org/v3-flatcontainer/errorprone.net.coreanalyzers/0.4.0-beta.1/errorprone.net.coreanalyzers.0.4.0-beta.1.nupkg OK https://api.nuget.org/v3-flatcontainer/errorprone.net.coreanalyzers/0.4.0-beta.1/errorprone.net.coreanalyzers.0.4.0-beta.1.nupkg 10ms Installed ErrorProne.NET.CoreAnalyzers 0.4.0-beta.1 from https://api.nuget.org/v3/index.json to /home/mediapipe/.nuget/packages/errorprone.net.coreanalyzers/0.4.0-beta.1 with content hash jW7IDd7xMJoz2IyhyI2NE7JB5o+mH9AuLHDh4I79dL/WpbLRhPGXlcK1GmZa2HjSXEl2a9zXQ4/Hyo9lOaNq1g==. Adding package 'ErrorProne.NET.CoreAnalyzers.0.4.0-beta.1' to folder '/home/mediapipe/.nuget' Added package 'ErrorProne.NET.CoreAnalyzers.0.4.0-beta.1' to folder '/home/mediapipe/.nuget' Restoring NuGet package ErrorProne.NET.Structs.0.4.0-beta.1. GET https://api.nuget.org/v3-flatcontainer/errorprone.net.structs/0.4.0-beta.1/errorprone.net.structs.0.4.0-beta.1.nupkg OK https://api.nuget.org/v3-flatcontainer/errorprone.net.structs/0.4.0-beta.1/errorprone.net.structs.0.4.0-beta.1.nupkg 9ms Installed ErrorProne.NET.Structs 0.4.0-beta.1 from https://api.nuget.org/v3/index.json to /home/mediapipe/.nuget/packages/errorprone.net.structs/0.4.0-beta.1 with content hash 3cTaYc/hcDJumpCTjYNu+X0IEyTFRKHsIlsHwDjju1rGpH7/9dcostXJlRtmIIno6PgRWIh5ZalHDq76QYUA2w==. Adding package 'ErrorProne.NET.Structs.0.4.0-beta.1' to folder '/home/mediapipe/.nuget' Added package 'ErrorProne.NET.Structs.0.4.0-beta.1' to folder '/home/mediapipe/.nuget' GET https://api.nuget.org/v3/vulnerabilities/index.json OK https://api.nuget.org/v3/vulnerabilities/index.json 7ms GET https://api.nuget.org/v3-vulnerabilities/2024.10.31.05.55.13/vulnerability.base.json OK https://api.nuget.org/v3-vulnerabilities/2024.10.31.05.55.13/vulnerability.base.json 10ms GET https://api.nuget.org/v3-vulnerabilities/2024.10.31.05.55.13/2024.11.02.05.55.20/vulnerability.update.json OK https://api.nuget.org/v3-vulnerabilities/2024.10.31.05.55.13/2024.11.02.05.55.20/vulnerability.update.json 8ms INFO (build.py): Downloaded protobuf dlls INFO (build.py): Building resource files WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/1c154bd755153b5c6ada4bbed58facf23f6abffc.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found WARNING: Download from http://mirror.tensorflow.org/github.com/bazelbuild/rules_closure/archive/cf1e44edb908e9616030cc83d085989b8e6cd6df.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/dcffa4094e13d56a30b2cdcdb709ce5d71b38953.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found INFO: Analyzed target //mediapipe_api:mediapipe_assets (6 packages loaded, 45 targets configured). INFO: Found 1 target... Target //mediapipe_api:mediapipe_assets up-to-date: bazel-bin/mediapipe_api/mediapipe_assets.zip INFO: Elapsed time: 1.672s, Critical Path: 0.44s INFO: 13 processes: 2 internal, 11 linux-sandbox. INFO: Build completed successfully, 13 total actions INFO (build.py): Built resource files INFO (build.py): Building native libraries for Android... WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/1c154bd755153b5c6ada4bbed58facf23f6abffc.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found WARNING: Download from http://mirror.tensorflow.org/github.com/bazelbuild/rules_closure/archive/cf1e44edb908e9616030cc83d085989b8e6cd6df.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/dcffa4094e13d56a30b2cdcdb709ce5d71b38953.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found INFO: Build options --copt, --cpu, --crosstool_top, and 4 more have changed, discarding analysis cache. WARNING: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/mediapipe/mediapipe/framework/BUILD:58:24: in cc_library rule @mediapipe//mediapipe/framework:calculator_cc_proto: target '@mediapipe//mediapipe/framework:calculator_cc_proto' depends on deprecated target '@com_google_protobuf//:cc_wkt_protos': Only for backward compatibility. Do not use. WARNING: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/mediapipe/mediapipe/framework/tool/BUILD:200:24: in cc_library rule @mediapipe//mediapipe/framework/tool:field_data_cc_proto: target '@mediapipe//mediapipe/framework/tool:field_data_cc_proto' depends on deprecated target '@com_google_protobuf//:cc_wkt_protos': Only for backward compatibility. Do not use. WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/pytorch/cpuinfo/archive/87d8234510367db49a65535021af5e1838a65ac2.zip failed: class java.io.FileNotFoundException GET returned 404 Not Found WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/google/XNNPACK/archive/b9d4073a6913891ce9cbd8965c8d506075d2a45a.zip failed: class java.io.FileNotFoundException GET returned 404 Not Found INFO: Analyzed target //mediapipe_api/java/com/github/homuler/mediapipe:mediapipe_android (169 packages loaded, 19498 targets configured). INFO: Found 1 target... INFO: From Executing genrule @org_tensorflow//tensorflow/lite/acceleration/configuration:configuration_schema: When you use --proto, that you should check for conformity yourself, using the existing --conform INFO: From Generating flatbuffer files for metadata_schema_cc_srcs: @mediapipe//mediapipe/tasks/metadata:metadata_schema_cc_srcs:

warning: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/mediapipe/mediapipe/tasks/metadata/metadata_schema.fbs:486: 21: warning: field names should be lowercase snake_case, got: sentencePiece_model

INFO: From Generating flatbuffer files for metadata_schema_cc_srcs: @mediapipe//mediapipe/tasks/metadata:metadata_schema_cc_srcs:

warning: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/external/mediapipe/mediapipe/tasks/metadata/metadata_schema.fbs:486: 21: warning: field names should be lowercase snake_case, got: sentencePiece_model

INFO: From Executing genrule @org_tensorflow//tensorflow/lite/acceleration/configuration:configuration_schema: When you use --proto, that you should check for conformity yourself, using the existing --conform INFO: From Compiling mediapipe/framework/deps/status_builder.cc [for tool]: external/mediapipe/mediapipe/framework/deps/status_builder.cc: In lambda function: external/mediapipe/mediapipe/framework/deps/status_builder.cc:97:3: warning: control reaches end of non-void function [-Wreturn-type] 97 | }()); | ^ INFO: From Compiling mediapipe/framework/tool/message_type_util.cc [for tool]: external/mediapipe/mediapipe/framework/tool/message_type_util.cc: In static member function 'static int mediapipe::DescriptorReader::MatchingPrefixLength(const string&, const string&)': external/mediapipe/mediapipe/framework/tool/message_type_util.cc:67:14: warning: comparison of integer expressions of different signedness: 'int' and 'const long unsigned int' [-Wsign-compare] 67 | while (i < std::min(s.size(), t.size()) && s[i] == t[i]) { | ^~~~~~~~ INFO: From Executing genrule //mediapipe_api/java/com/github/homuler/mediapipe:mediapipe_android: Archive: /home/mediapipe/.cache/bazel/_bazel_mediapipe/6c11c387a04fa8f17c87a5c6db6ee555/sandbox/linux-sandbox/5404/execroot/mediapipe_api/bazel-out/arm64-v8a-opt/bin/mediapipe_api/java/com/github/homuler/mediapipe/mediapipe_android_dummy_app_unsigned.apk creating: lib/arm64-v8a/ inflating: lib/arm64-v8a/libopencv_java3.so inflating: lib/arm64-v8a/libmediapipe_jni.so creating: lib/armeabi-v7a/ inflating: lib/armeabi-v7a/libopencv_java3.so inflating: lib/armeabi-v7a/libmediapipe_jni.so adding: jni/arm64-v8a/libmediapipe_jni.so (deflated 75%) adding: jni/arm64-v8a/libopencv_java3.so (deflated 59%) adding: jni/armeabi-v7a/libmediapipe_jni.so (deflated 74%) adding: jni/armeabi-v7a/libopencv_java3.so (deflated 50%) Target //mediapipe_api/java/com/github/homuler/mediapipe:mediapipe_android up-to-date: bazel-bin/mediapipe_api/java/com/github/homuler/mediapipe/mediapipe_android.aar INFO: Elapsed time: 2353.271s, Critical Path: 663.92s INFO: 5555 processes: 506 internal, 5041 linux-sandbox, 8 worker. INFO: Build completed successfully, 5555 total actions INFO (build.py): Built native libraries for Android INFO (build.py): Installing built resources... INFO (build.py): Installed`

its still giving me the same error

DMLV-dev commented 2 days ago

Screenshot_2024-11-02_230252

I have this error

homuler commented 1 day ago

How about specifying the --android_ndk_api_level?

python build.py build --android arm64 --android_fat_apk_cpu=armeabi-v7a,arm64-v8a --android_ndk_api_level 21 --solutions=face_mesh

If you want to run the Pose solution, please specify pose in solutions.

DMLV-dev commented 1 day ago

Hi @homuler It's all working now. thank you very much. but the skeleton detection is lagging. its like delayed. is there any setting fixes for this? and also question. do I change the solutions=holistic if I want to use the holistic scene?

homuler commented 15 hours ago

If you're running the sample scene, try changing the running mode from LIVE_STREAM to VIDEO (or IMAGE) Screenshot_20241104_125824

and setting canUseGpuImage to false (rewrite the code) (cf. https://github.com/homuler/MediaPipeUnityPlugin/issues/1238#issuecomment-2335017929). https://github.com/homuler/MediaPipeUnityPlugin/blob/c691e7c85bcffd7e669fe71a08e86dab77bf6b75/Assets/MediaPipeUnity/Samples/Scenes/Pose%20Landmark%20Detection/PoseLandmarkerRunner.cs#L76-L77

do I change the solutions=holistic if I want to use the holistic scene?

Yes, but if you want to run both scenes, you can specify both (--solutions=pose,holistic).