Start Video no response

[thenighthunter0]

hi, when I run mediapipe4u-demo,start image success,

but start video,no response. Are there any restrictions on input video？

[endink]

First of all , vedio capture require GStreamer (Because there are too many files in Gstreamer, and there is no official way to crop files, I did not include it in the plugin) . You need install it by yourself. And if you package your project, gstreamer will automatically copy into your packaging output.

For learning how to install GStreamer, please see MediaPipe4U document.

[thenighthunter0]

I have followed MediaPipe4U document. can I find any other information, like logs,when start video? maybe I need to reboot computer? I will try.

[endink]

maybe is same issue #5

To work hard you follow these steps：

1. Download this version: https://cowtransfer.com/s/1c4f0dcd0b9a40
2. Run demo project in UnrealEngine edtior, copy your error log information in Unreal Edtiro and paste to here.

It's my fault, sorry ...

[endink]

BTW，In general, you should see the error log in the Console viewer (UnrealEngine Editor). Isn't there any error log printed?

[thenighthunter0]

BTW，In general, you should see the error log in the Console viewer (UnrealEngine Editor). Isn't there any error log printed?

I will try ,thanks :)

yes,some errors as follows

LogMediaPipeGStreamer: Display: GStreamer is started. filesrc location="../../../../../../data/falldown_video_split/falldown_00.mp4" ! decodebin ! videoconvert ! video/x-raw,format=(string)RGBA ! videoconvert ! appsink name=mediapipe_sink LogGStreamer: Error: GstPipeline BUS ERROR: Resource not found. LogMediaPipe: Display: [mediapipe_api.dll] CalculatorGraph::StartRun enable_segmentation : mediapipe::Packet with timestamp: Timestamp::Unset() and type: bool input_horizontally_flipped : mediapipe::Packet with timestamp: Timestamp::Unset() and type: bool input_rotation : mediapipe::Packet with timestamp: Timestamp::Unset() and type: int input_vertically_flipped : mediapipe::Packet with timestamp: Timestamp::Unset() and type: bool model_complexity : mediapipe::Packet with timestamp: Timestamp::Unset() and type: int refine_face_landmarks : mediapipe::Packet with timestamp: Timestamp::Unset() and type: bool smooth_landmarks : mediapipe::Packet with timestamp: Timestamp::Unset() and type: bool smooth_segmentation : mediapipe::Packet with timestamp: Timestamp::Unset() and type: bool use_prev_landmarks : mediapipe::Packet with timestamp: Timestamp::Unset() and type: bool LogMediaPipe: Display: [mediapipe_api.dll] ------------> Start Loop Work Thread <------------

[endink]

Oh~~ its not issue #5 .

This is a known problem, it about you mp4 file path. I'm guessing you have special characters in your file path, such as dots, commas, etc. To fix it, you need keep your folder and file names simple, preferably only contain letters, numbers, underscores, lines, it can have unicode characters (such as Chinese), but can not have special symbols, GStreamer needs to use some special symbols, I dont know how to escape these character currently, because GStreamer has no document about it.

[thenighthunter0]

filesrc location="../../../../../../dongbu/MediaPipe4U-Demo/1.mp4" ! decodebin ! videoconvert ! video/x-raw,format=(string)RGBA ! videoconvert ! appsink name=mediapipe_sink LogGStreamer: Error: GstPipeline BUS ERROR: Resource not found.

where is current dir?

[endink]

emmm...

Do it:

Open FileHelper.cpp in Demo project.

search every line (there are 2 lines) in this file, modify:

File = files[0];

to

File = FPaths::ConvertRelativePathToFull(files[0]);

[endink]

If this doesn't work, put your mp4 file in a different folder, not in project folder, this is a newly discovered bug ~~~

[endink]

I will test for it~~~ so, sorry

[thenighthunter0]

File = FPaths::ConvertRelativePathToFull(files[0]);

thanks for your reply:), you do a good job.

[thenighthunter0]

I will test for it~~~ so, sorry

it work, 我试图仅通过，插件，达成虚拟人物视频数据制作。现在有两个问题： 1.摔倒不是在空中，有根运动， 2.需要导出ue动画资产，驱动其他的metahuman

需要学习 ue动画蓝图相关吗？你有什么建议吗？这两周我都有时间花在这问题上。

我的解决策略如下： 1.研究demo，获得动画序列，再用sequence加载其他不同的meta-human，录制视频。

BTW,meta-human的蓝图人物，上本身，下半身是分开的，demo不好直接驱动啊，meta-human的basebone可以按你教程驱动，如下图：

[endink]

1. 根运动问题，查看 Demo Project 的 Repo 里的 issue， 简单说问题不在于根运动，我可以很容易支持根运动，问题在于位移不准确和标定需要非常麻烦的流程。
2. Metahuman 是插件天然支持的，Preset 设为 UE5 即可直接驱动 Metahuman, 其他网友用我的插件制作的效果：https://www.bilibili.com/video/BV1Nt4y1N7P4

metahuman 怎么变成可驱动的 character UE 已经给出了非常详尽的文档：

https://docs.metahuman.unrealengine.com/zh-CN/retargeting-animations-to-a-metahuman-at-runtime

对于不需要运行时加载模型（例如让用户选择一个 FBX 然后加载模型，这需要大量编程来处理 shader 和 材质问题）的场景，我建议使用 UE5 骨骼 （不是隐藏一个小金人，而是使用 Lyra 项目类似的方式，而应该放置一个没有顶点的模型在场景中以达到最佳性能，这个模型因为没有顶点，不会浪费渲染性能，而且完全不可见）IKRetarget 或者直接使用插件的 MediaPipeRemapAsset，具体选什么看你要支持些什么，性能角度看，优先使用 MediaPipeRemapAsset, 工作量也小不少。

[endink]

如果对 UE 非常熟练，这个插件可以驱动任意模型，如果你不理解 metahuman “分离”的设计或者无法处理它，我认为是你不够熟悉它，请多阅读文档，任何组件都需要你仔细阅读文档，如果没有其他问题我将关闭这个 issue，感谢反馈~~

[thenighthunter0]

感谢，明白了许多。

还有一个问题是，如何通过动补得到ue动画序列？这方面你有什么好的建议吗？比如需要学习哪些知识？

目前我只会，IK, IKRetarget,你的相关教程文档（除了蓝图动补那些），已经会Mixamo到meta-human,然后IKRetarget得到动画序列，最后在sequence里面，meta-human使用动画序列，制作出我要的不同虚拟人的摔倒动作。

[thenighthunter0]

如果对 UE 非常熟练，这个插件可以驱动任意模型，如果你不理解 metahuman “分离”的设计或者无法处理它，我认为是你不够熟悉它，请多阅读文档，任何组件都需要你仔细阅读文档，如果没有其他问题我将关闭这个 issue，感谢反馈~~

嗯嗯，好的

[endink]

UE 的工具链我不是太熟，或许 Take Recorder 是你需要的东西~