wongfei / ue4-mediapipe-plugin

UE4 MediaPipe plugin
Apache License 2.0
294 stars 99 forks source link

[QUESTION] Can you give some info of how to implement another models? #5

Closed iMaciaP closed 3 years ago

iMaciaP commented 3 years ago

Thank you for the plugin, very nice job!.

As i ask, can you give some details or docs of how to implement another model like "objectron" from mediapipe into the plugin with c++? Best regards.

wongfei commented 3 years ago

For some unknown reason objectron examples don't work on my desktop, consider it unsupported right now..

But theoretically plugin is able to run objectron pipeline.

Check:

https://github.com/wongfei/ue4-mediapipe-plugin/blob/master/Plugins/MediaPipe/ThirdParty/mediapipe/Data/mediapipe/unreal/objectron_landmarks.pbtxt

==

For other packets (not landmarks) just implement IUmpPacketCallback::OnUmpPacket:

https://github.com/wongfei/ue4-mediapipe-plugin/blob/master/Plugins/MediaPipe/Source/MediaPipe/Private/ump_api.h#L84

https://github.com/wongfei/ue4-mediapipe-plugin/blob/master/Plugins/MediaPipe/Source/MediaPipe/Private/MediaPipeLandmarkObserverComponent.cpp#L79

https://github.com/wongfei/ue4-mediapipe-plugin/blob/master/Plugins/MediaPipe/Source/MediaPipe/Private/MediaPipeLandmarkObserverComponent.cpp#L74