App hangs on Android on Motorola E6

gb2111 commented 1 year ago

Plugin Version or Commit ID

0.11.0

Unity Version

2021.3.18

Your Host OS

Windows

Target Platform

Android

Target Device

Motorola E6

[Windows Only] Visual Studio C++ and Windows SDK Version

No response

[Linux Only] GCC/G++ and GLIBC Version

No response

[Android Only] Android Build Tools and NDK Version

No response

[iOS Only] XCode Version

No response

Build Command

n/a

Bug Description

Sample app hangs. I realize that phone is really a potato but I wonder if we could get around and either not allow pose tracking or just handle without hanging the system.

In Sync mode it hangs when calling

UnsafeNativeMethods.mp_OutputStreamPoller__Next_Ppacket(mpPtr, packet.mpPtr, out var result).Assert(); Mediapipe.OutputStreamPoller1:Next(Packet1) Mediapipe.Unity.OutputStream2:TryGetNext(TValue&, Int64, Boolean) Mediapipe.Unity.GraphRunner:TryGetNext(OutputStream2, TValue&, Boolean, Int64)

In Async the camera works compared to Sync but when I enter mediapipe settings, change e.g. model and close it also hangs in function

UnsafeNativeMethods.mp_CalculatorGraph__WaitUntilDone(mpPtr, out var statusPtr; Mediapipe.Unity.GraphRunner:Stop()

Thanks.

Steps to Reproduce the Bug

In Async mode just enter settings, change model to Light and clsoe settings.

In Sync method app will not display any feed from the camera.

Log

C:\Program Files\Unity\Hub\Editor\2022.1.2f1\Editor\Data\PlaybackEngines\AndroidPlayer\SDK\platform-tools>adb logcat Unity:V native:V tflite:V CRASH:E AndroidRuntime:E "*:S" --------- beginning of main --------- beginning of system 04-19 10:01:21.848 30143 30158 I Unity : MemoryManager: Using 'Dynamic Heap' Allocator. 04-19 10:01:21.889 30143 30158 I Unity : SystemInfo CPU = ARM64 FP ASIMD AES, Cores = 8, Memory = 3777mb 04-19 10:01:21.889 30143 30158 I Unity : SystemInfo ARM big.LITTLE configuration: 4 big (mask: 0xf), 4 little (mask: 0xf0) 04-19 10:01:21.890 30143 30158 I Unity : ApplicationInfo com.DefaultCompany.MediaPipeUnityPlugin version 0.1 04-19 10:01:21.890 30143 30158 I Unity : Built from '2021.3/staging' branch, Version '2021.3.18f1 (3129e69bc0c7)', Build type 'Release', Scripting Backend 'il2cpp', CPU 'arm64-v8a', Stripping 'Enabled' 04-19 10:01:22.140 30143 30158 I Unity : Company Name: DefaultCompany 04-19 10:01:22.140 30143 30158 I Unity : Product Name: MediaPipeUnityPlugin 04-19 10:01:22.189 30143 30158 D Unity : GL_EXT_debug_marker GL_ANDROID_extension_pack_es31a GL_EXT_blend_minmax GL_EXT_buffer_storage GL_EXT_clear_texture GL_EXT_clip_control GL_EXT_color_buffer_float GL_EXT_color_buffer_half_float GL_EXT_conservative_depth GL_EXT_copy_image GL_EXT_discard_framebuffer GL_EXT_draw_buffers GL_EXT_draw_buffers_indexed GL_EXT_draw_elements_base_vertex GL_EXT_EGL_image_array GL_EXT_external_buffer GL_EXT_float_blend GL_EXT_geometry_point_size GL_EXT_geometry_shader GL_EXT_gpu_shader5 GL_EXT_memory_object GL_EXT_memory_object_fd GL_EXT_multi_draw_arrays GL_EXT_multisampled_render_to_texture GL_EXT_multisampled_render_to_texture2 GL_EXT_occlusion_query_boolean GL_EXT_polygon_offset_clamp GL_EXT_primitive_bounding_box GL_EXT_pvrtc_sRGB GL_EXT_read_format_bgra GL_EXT_robustness GL_EXT_separate_shader_objects GL_EXT_shader_framebuffer_fetch GL_EXT_shader_group_vote GL_EXT_shader_implicit_conversions GL_EXT_shader_io_blocks GL_EXT_shader_non_constant_global_initializers GL_EXT_shader_pixel_local_storage GL_EXT_shaderpixel 04-19 10:01:22.189 30143 30158 D Unity : local_storage2 GL_EXT_shader_texture_lod GL_EXT_sparse_texture GL_EXT_sRGB_write_control GL_EXT_tessellation_point_size GL_EXT_tessellation_shader GL_EXT_texture_border_clamp GL_EXT_texture_buffer GL_EXT_texture_cube_map_array GL_EXT_texture_filter_anisotropic GL_EXT_texture_format_BGRA8888 GL_EXT_texture_rg GL_EXT_texture_sRGB_decode GL_EXT_texture_sRGB_R8 GL_EXT_texture_sRGB_RG8 GL_EXT_YUV_target GL_IMG_bindless_texture GL_IMG_framebuffer_downsample GL_IMG_multisampled_render_to_texture GL_IMG_program_binary GL_IMG_read_format GL_IMG_shader_binary GL_IMG_texture_compression_pvrtc GL_IMG_texture_compression_pvrtc2 GL_IMG_texture_format_BGRA8888 GL_IMG_texture_npot GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent GL_KHR_debug GL_KHR_robustness GL_KHR_texture_compression_astc_ldr GL_OES_compressed_ETC1_RGB8_texture GL_OES_depth24 GL_OES_depth_texture GL_OES_draw_buffers_indexed GL_OES_draw_elements_base_vertex GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_EGL_image_externalessl3 GL 04-19 10:01:22.189 30143 30158 D Unity : OES_EGL_sync GL_OES_element_index_uint GL_OES_fragment_precision_high GL_OES_geometry_point_size GL_OES_geometry_shader GL_OES_get_program_binary GL_OES_gpu_shader5 GL_OES_mapbuffer GL_OES_packed_depth_stencil GL_OES_required_internalformat GL_OES_rgb8_rgba8 GL_OES_sample_shading GL_OES_sample_variables GL_OES_shader_image_atomic GL_OES_shader_io_blocks GL_OES_shader_multisample_interpolation GL_OES_standard_derivatives GL_OES_surfaceless_context GL_OES_tessellation_point_size GL_OES_tessellation_shader GL_OES_texture_border_clamp GL_OES_texture_buffer GL_OES_texture_cube_map_array GL_OES_texture_float GL_OES_texture_half_float GL_OES_texture_npot GL_OES_texture_stencil8 GL_OES_texture_storage_multisample_2d_array GL_OES_vertex_array_object GL_OES_vertex_half_float 04-19 10:01:24.664 30143 30158 I native : I20230419 10:01:24.664314 30158 gl_context_egl.cc:84] Successfully initialized EGL. Major : 1 Minor: 4 04-19 10:01:24.670 30143 30199 I native : I20230419 10:01:24.670215 30199 gl_context.cc:342] GL version: 3.2 (OpenGL ES 3.2 build 1.10@5130912) 04-19 10:01:25.360 30143 30158 I native : I20230419 10:01:25.359982 30158 gl_context_egl.cc:84] Successfully initialized EGL. Major : 1 Minor: 4 04-19 10:01:25.365 30143 30308 I native : I20230419 10:01:25.365548 30308 gl_context.cc:342] GL version: 3.2 (OpenGL ES 3.2 build 1.10@5130912) 04-19 10:01:25.404 30143 30199 I tflite : Initialized TensorFlow Lite runtime. 04-19 10:01:25.418 30143 30199 I tflite : Replacing 291 node(s) with delegate (unknown) node, yielding 1 partitions. 04-19 10:01:25.713 30143 30190 E Unity : OPENGL NATIVE PLUG-IN ERROR: GL_INVALID_ENUM: enum argument out of range 04-19 10:01:25.997 30143 30199 I tflite : Replacing 291 node(s) with delegate (unknown) node, yielding 1 partitions.

Screenshot/Video

No response

Additional Context

No response

homuler commented 1 year ago

First, please build the app with Development build enabled and share the log.

I realize that phone is really a potato but I wonder if we could get around and either not allow pose tracking or just handle without hanging the system.

Can you confirm that the official sample app (provided by MediaPipe developers) runs on your phone?

In Async mode just enter settings, change model to Light and clsoe settings. In Sync method app will not display any feed from the camera.

Please share the video.

In Sync mode it hangs when calling

UnsafeNativeMethods.mp_OutputStreamPoller__Next_Ppacket(mpPtr, packet.mpPtr, out var result).Assert(); Mediapipe.OutputStreamPoller1:Next(Packet1) Mediapipe.Unity.OutputStream2:TryGetNext(TValue&, Int64, Boolean) Mediapipe.Unity.GraphRunner:TryGetNext(OutputStream2, TValue&, Boolean, Int64)

How did you confirm that? It looks to me that the above is a stack trace, but there's no such output in the log you shared.

gb2111 commented 1 year ago

Hello. I am attaching log from Development build. This time app crashes regularly quickly after something is first time detected as you can see in the video. In Sync mode it is black until mediapipe start processing when we start seeing camera feed and when it first time detect a pose it crashes. In Async mode it is same except we see camera immediately. log.txt

Official mediapipe app is initially black and it start working after 3 minutes and works with 1fps. But this is not the very latest app so I might also try latest one. .

I confirmed exact place where code is blocked, the native function calls, by adding extra logs to the code that is normally not available in sample app. I tested it on Async and Sync mode. It is not visible in the stack trace cause normally nothing is logged there and there is no unhandled exception.

Here is video:

https://user-images.githubusercontent.com/22025896/233057113-bdda06fd-2e5d-4b1d-a243-f14e809b5b36.mp4

homuler commented 1 year ago

Couldn't you get the logs when the app crashed using adb? I think the filter may be too strong and the logs may not included in the log.txt. For example, is it possible that the app was killed by the OS because it was consuming a lot of memory?

gb2111 commented 1 year ago

I am using adb logcat with filter you have provided. Do you have perhaps any suggestion for a filter? Cause I can provide everything but as I could see there is ton's of logs and might be hard to capture. Are you ok to send logs wiht everything instead?

homuler commented 1 year ago

If you want to identify the cause, I believe someone needs to read the logs. And I think the app developer should be the one to read them.

gb2111 commented 1 year ago

Hello. Please find logs with no filter from Async mode run that ends with crash. log.txt

gb2111 commented 1 year ago

Hi @homuler, Will u be looking on that bug?

homuler commented 1 year ago

As I mentioned before, isn't it just that the app was KILLED by the OS due to lack of memory?

04-20 08:11:38.728   946  1042 W InputDispatcher: channel '7e11d6a com.DefaultCompany.MediaPipeUnityPlugin/com.unity3d.player.UnityPlayerActivity (server)' ~ Consumer closed input channel or an error occurred.  events=0x9
04-20 08:11:38.729   500   500 I ADB_SERVICES: for fd 19, revents = 2011
04-20 08:11:38.729   946  1042 E InputDispatcher: channel '7e11d6a com.DefaultCompany.MediaPipeUnityPlugin/com.unity3d.player.UnityPlayerActivity (server)' ~ Channel is unrecoverably broken and will be disposed!
04-20 08:11:38.729   500   500 I ADB_SERVICES: for fd 19, revents = 2011
04-20 08:11:38.730   452   503 I SurfaceFlinger: [SF client] Remove(0x75aa592900) for (64:)
04-20 08:11:38.732   946  1356 I ActivityManager: Process com.DefaultCompany.MediaPipeUnityPlugin (pid 12214) has died: fore TOP
04-20 08:11:38.734   946  1356 W ActivityManager: Force removing ActivityRecord{a7d00f1 u0 com.DefaultCompany.MediaPipeUnityPlugin/com.unity3d.player.UnityPlayerActivity t135}: app died, no saved state
04-20 08:11:38.750   529 12299 D isp_tuning_custom: [ratioMapping] sensorId = 30c7
04-20 08:11:38.752   529 12299 W MtkCam/ImgBuf: [IonHeap::onCreate] IonHeap@ Bad stride at 0-th plane: heapStridesInBytes(2768) -> (1476.266724)(pixels) -> StridesInBytes(2767)
04-20 08:11:38.760   946   964 W libprocessgroup: kill(-12214, 9) failed: No such process
04-20 08:11:38.761   946   964 I libprocessgroup: Successfully killed process cgroup uid 10114 pid 12214 in 28ms
04-20 08:11:38.769   452   977 D GuiExt  : [GuiExtS] binder of dump tunnel(BQM-[12214:com.DefaultCompany.MediaPipeUnityPlugin]) died

P.S. I may have worded it in a misleading way, but I am of the opinion that you, the person in trouble, should read the log. That is, please don't just throw logs at me without reading them.

gb2111 commented 1 year ago

Ok. You wrote that dev can read logs and since you are dev of that software I shared all logs :) Which I checked before sending you by the way same as debugged as much as I could. Sorry, I have missunderstood your intention and thank for your help.

homuler commented 1 year ago

Thanks for sharing the raw logs without filtering (if filtered, I can't say anything with confidence).

However, if you find it hard to read the log, it is more or less the same for everyone. I hope next time you will at least write about the research you have done on the possibilities I have raised.

For example, is it possible that the app was killed by the OS because it was consuming a lot of memory?

In this case, you could have found the log by searching for the word "kill".

homuler / MediaPipeUnityPlugin