UE5.2 support

[jlightfoot]

Hi. I'm trying to animate a Metahuman with body tracking in k4A. I haven't been able to get your plugin running in UE5.2, however. I get "Plugin 'AzureKinect' failed to load because module 'AzureKinect' could not be loaded. There may be an operating system error or the module may not be properly set up." on project load. Any ideas?

[cormacmadden]

Hey sorry for the late reply. I think its to do with files in the plugin binaries folder. I have it working in 5.2 atm I'll try update the repo today and let you know what the problem is.

[jlightfoot]

Thanks for the response. I built the plugin manually and cleaned up a bunch of compilation errors but it looks like all I really needed to do to get it to build on project load was add C:\Program Files\Azure Kinect SDK v1.4.1\sdk\windows-desktop\amd64\release\bin and/or (?) C:\Program Files\Azure Kinect Body Tracking SDK\sdk\windows-desktop\amd64\release\bin to my path. Once in the project, I'm able to start the camera but get a bunch of these errors in the output log: AzureKinectDeviceLog: Error: Couldn't get Body Frame: Failed to enqueue capture to tracker!.

[cormacmadden]

Hey, I got the same error as you when I tried the method of editing the path variable. Instead I've just added a bunch of the dlls to this folder. It's not an ideal solution but it works for me. I'm going to look into a cleaner solution that imports them automatically, but try this for now. All the files there can be found in the Program Files folders. (I needed to remove the path variables to get it to work again).

[cormacmadden]

Don't bother adding all the .h or .lib files, it works fine without them.

[cormacmadden]

I think if you just add C:\Program Files\Azure Kinect Body Tracking SDK\sdk\netstandard2.0\release as an environment PATH variable alongside the other two you mentioned, it should work. Let me know how that goes. Also be aware that the skeleton might glitch through the floor so move the body up off the ground before testing.

[jlightfoot]

Ok, so my test project now loads fine with the plugin enabled. When I start the camera, though, I get the following errors. And the mannequin doesn't move (all I did to the mannequin was retarget the skeleton and drop it into my scene). Is there something I'm doing wrong?

LogOutputDevice: Error: === Handled ensure: === LogOutputDevice: Error: Ensure condition failed: oldValue == newValue [File:D:\build++UE5\Sync\Engine\Source\Runtime\Core\Private\HAL\ThreadingBase.cpp] [Line: 310] LogOutputDevice: Error: oldValue(1) newValue(0) If this check fails make sure that there is a FTaskTagScope(ETaskTag::EParallelRenderingThread) as deep as possible on the current callstack, you can see the current value in ActiveNamedThreads(0), GRenderingThread(839fb40), GIsRenderingThreadSuspended(0) LogOutputDevice: Error: Stack: LogOutputDevice: Error: [Callstack] 0x00007ffb04f55302 UnrealEditor-Core.dll!UnknownFunction [] LogOutputDevice: Error: [Callstack] 0x00007ffb0181548a UnrealEditor-Engine.dll!UnknownFunction [] LogOutputDevice: Error: [Callstack] 0x00007ffad3088d1b UnrealEditor-AzureKinect.dll!UAzureKinectDevice::CaptureBodyIndexImage() [C:\Users\username\Documents\Unreal Projects\Kinect_Test\Plugins\Kinect-Body-Tracking-Unreal-Engine-Plugin-main\Source\AzureKinect\Private\AzureKinectDevice.cpp:577] LogOutputDevice: Error: [Callstack] 0x00007ffad3093229 UnrealEditor-AzureKinect.dll!UAzureKinectDevice::UpdateSkeletons() [C:\Users\username\Documents\Unreal Projects\Kinect_Test\Plugins\Kinect-Body-Tracking-Unreal-Engine-Plugin-main\Source\AzureKinect\Private\AzureKinectDevice.cpp:626] LogOutputDevice: Error: [Callstack] 0x00007ffad3092fdf UnrealEditor-AzureKinect.dll!UAzureKinectDevice::UpdateAsync() [C:\Users\username\Documents\Unreal Projects\Kinect_Test\Plugins\Kinect-Body-Tracking-Unreal-Engine-Plugin-main\Source\AzureKinect\Private\AzureKinectDevice.cpp:310] LogOutputDevice: Error: [Callstack] 0x00007ffad30918c9 UnrealEditor-AzureKinect.dll!FAzureKinectDeviceThread::Run() [C:\Users\username\Documents\Unreal Projects\Kinect_Test\Plugins\Kinect-Body-Tracking-Unreal-Engine-Plugin-main\Source\AzureKinect\Private\AzureKinectDeviceThread.cpp:43] LogOutputDevice: Error: [Callstack] 0x00007ffb054c2162 UnrealEditor-Core.dll!UnknownFunction [] LogOutputDevice: Error: [Callstack] 0x00007ffb054b81b0 UnrealEditor-Core.dll!UnknownFunction [] LogOutputDevice: Error: [Callstack] 0x00007ffb964b26ad KERNEL32.DLL!UnknownFunction [] LogOutputDevice: Error: [Callstack] 0x00007ffb9696aa68 ntdll.dll!UnknownFunction []

[fhqht12331]

What should each of the three PATH system environment variables be named? Do you have a more detailed explanation?

[cormacmadden]

What should each of the three PATH system environment variables be named? Do you have a more detailed explanation?

Hey, your environment variables should look as they do in this image. Hopefully that clears it up.

[cormacmadden]

Ok, so my test project now loads fine with the plugin enabled. When I start the camera, though, I get the following errors. And the mannequin doesn't move (all I did to the mannequin was retarget the skeleton and drop it into my scene). Is there something I'm doing wrong?

Hey, I have gotten that oldValue == newValue error message before but I can't remember how I fixed it. Try deleting the Binaries and Intermediate folders from the Plugins folder and opening the project again. Make sure your Kinect isn't already on when you try use it in Unreal (white light is off).

To get the skeleton Tracking in Unreal you just need to add the ABP_AureKinectSkeletonToHumanoid and the BP_AzureKinectActor to the scene. If the ABP_AureKinectSkeletonToHumanoid doesn't have a Skeletal Mesh Asset, you can set it to the SK_Mannequin from the third person starter assets or the TutorialTPP asset. Set the Skeletal Mesh Actor parameter on the BP_AzureKinectActor to the ABP_AureKinectSkeletonToHumanoid we placed in the scene, and it should work when you hit play.

I'm currently working on getting the plugin to work with multiple people in the scene.

Let me know if any of that works.

[jlightfoot]

Still no luck. I followed your instructions and now the mannequin just disappears when a run the scene. I'm still getting that series of errors when the camera starts, by the way. I'll see if I can start fresh with a empty project and give it another go.

[fhqht12331]

What should each of the three PATH system environment variables be named? Do you have a more detailed explanation?

Hey, your environment variables should look as they do in this image. Hopefully that clears it up.

Thanks to reply! I can use it but I can't see .uasset files in my project. How can I see .uasset files?

[fhqht12331]

oh! I solved it! I downloaded each .uasset files and pasted to my project folder-Plugins-Content

[cormacmadden]

That's great that worked fhqht12331! jlightfoot did that work in an empty project?

[jlightfoot]

Hey, Cormac, thanks for the follow-up. I did actually get it working. The location and rotation are wonky, but increasing both Z values to 90 seems to have solved that problem. It's also very jittery. I know the Kinect is very sensitive to light noise, so I'll see if I can control the lighting better and retry. Thanks for working on this!

[fhqht12331]

The assets are OK, but when you select and start the device, the render targets are all black. How can I fix it?

[cjpict]

How can i retarget with metahuman? Who's can do it?

[cjpict]

@jlightfoot Did it work well with meta-human? If you succeed, can you share how you did it?

[cjpict]

Hello, @cormacmadden The skeleton mesh trembles very badly every time you move. How can I fix it?

[Xuxiaoyan226]

嘿，科马克，谢谢你的跟进。我确实让它工作了。位置和旋转是不稳定的，但将两个 Z 值都增加到 90 似乎已经解决了这个问题。它也非常紧张。我知道 Kinect 对光噪非常敏感，所以我会看看我是否可以更好地控制照明并重试。感谢您为此工作！

Hello, how do you solve the problem that the character model cannot move normally when running?