search

nama-gatsuo / AzureKinectForUE

Exposes Azure Kinect Support for integration into Unreal Engine Applications.
91 stars 17 forks source link

Blueprint in BP_AzureKinect Actor does not compile. #1

Closed 120decibel closed 3 years ago

120decibel commented 3 years ago

I get the following errors when trying to use the BP_AzureKinect Actor:

kind regards Thomas

nama-gatsuo commented 3 years ago

Hi,

Thanks for pointing it out. This problem came from mismatch of codes and the blueprint node. Source code was updated but blueprint assets remained not.

In general right-clicking on the Anim node and "Refresh Nodes" will help somehow.

image

I just updated assets so please try pull master branch. https://github.com/nama-gatsuo/AzureKinectForUE/commit/1f212ccc53ffe5aa4d942ac2bd47760e95b7e0ce

120decibel commented 3 years ago

Hi,

Thank you for the swift reply.

I pulled the master, the BP compiles just fine now but once I start the game the enginge chrashes with the following:

If I place the Kinect Aktor in the scene and hit play the engine crashes with the following:

Unhandled Exception: EXCEPTION_ACCESS_VIOLATION writing address 0x0000000000000028

ntdll UE4Editor_AzureKinect!UAzureKinectDevice::GetNumTrackedSkeletons() [...\Unreal Projects\Kinect3\Plugins\AzureKinectForUE\Source\AzureKinect\Private\AzureKinectDevice.cpp:163] UE4Editor_AzureKinect!UAzureKinectDevice::execGetNumTrackedSkeletons() [...\Unreal Projects\Kinect3\Plugins\AzureKinectForUE\Intermediate\Build\Win64\UE4Editor\Inc\AzureKinect\AzureKinectDevice.gen.cpp:147]

I checked your code and it looks fine. I'm not sure if I'm using the plugin correctly though.

The Niagara PS works just fine btw (at least in the Editor) it doesn't show once I hit play.

cheers Thomas

120decibel commented 3 years ago

Not sure if this might be an issue with the Body Tracking SDK which version are us using there?

nama-gatsuo commented 3 years ago

Okay, that is... a bug. I have no idea why I could go through before... Just I can reproduce the same crash and fix it.

The problem was caused by thread locking without Kinect device thread opened as well as FCriticalSection . Could you try the latest?

Body Tracking SDK is v1.1.0. I think now you can get it work.

120decibel commented 3 years ago

Hi... sooo I just upgraded to Body Tracking SDK 1.1.0, Plugin Builds fine but no I can't start the Kinect anymore in the device details.

UE Log sais:

[2021.10.20-21.30.00:263][781]AzureKinectDeviceLog: Error: Cant't open: Failed to create k4abt tracker!

k4abt.log sais:

[2021-10-20 23:29:59.741] [error] [t=22872] [K4ABT] ..\src\TrackerHost\TrackerHost.cpp (185): Create(). Exception during initialization: C:\source\ort-restore\onnxruntime\onnxruntime\core\providers\dml\DmlExecutionProvider\src\MLOperatorAuthorImpl.cpp(1487)\onnxruntime.dll!00007FFCB260F2AC: (caller: 00007FFCB265EE66) Exception(3) tid(5958) 80070057 wrong parameter.

[2021-10-20 23:29:59.741] [error] [t=22872] [K4ABT] ..\src\sdk\k4abt.cpp (38): tracker->Create(sensor_calibration, config) returned failure in k4abt_tracker_create()

k4a.log shows no errors...

nama-gatsuo commented 3 years ago

Okay getting there... What do configs in AzureKinectDevice (in /AzureKinect Content in Content Browser) look like?

It should look like this. Both depth and color capture should be needed. image

And be sure:

YongsunLee commented 3 years ago

Hi,

I identified this issue yesterday, read the answer and pulled the master branch. However, the correct answer of this node cannot be inferred, leaving a question.

image

I want to know which asset to compare with.

image

Finally, I find that the StartDevicefunction through this plugin works fine.

It's just that BodyTracking doesn't work

120decibel commented 3 years ago

Hi

I think the issue for me is the upgrade to kinect body tracking SDK 1.1.0 Once I deactivate Skeleton tracking I can start the device.

kinect .

cheers Thomas

nama-gatsuo commented 3 years ago

Hi, @YongsunLee NOT EQUAL node represents just null check. So leaving empty should be okay.

If you want to play a SkeletalMesh by body tracking, you need to

image

Any SkeletalMesh would work if you create new Anim BP in which you can map bones via Azure Kinect Pose Anim node. Also need passing Azure Kinect Skeleton object in the same way BP_AzureKinect does to ABP_AzureKinectSkeletonToHumanoid.

Actually your project already succeeds to capture skeleton from AzureKinect because BodyIndexTexture is visible there. So things you need is to feed Azure Kinect Skeleton object to AzureKinectPose in AnimGraph.

YongsunLee commented 3 years ago

Thank you. @nama-gatsuo

image

This solved the problem.

The environment I'm running is as follows: 1) Azure Kinect SDK v1.4.1 2) Azure Kinect Body Tracking SDK 1.1.0 3) Using UE 4.27.1

Thanks you comment.

120decibel commented 3 years ago

Hi,

I got everything to work, I'm not sure why I had two ABP_AureKinectSceletonToHumanoid in the drop down for Anim Class but apparently I hat to select the second one.

The initial issue with the body tracker not starting was solved by placing the dlls of the tracking sdk into the binary folder of the engine. If I find the time I'll write up a small how to over the weekend.

GREAT WORK Nama Gastuo and a great PLUG IN!! Thank you very much!!

I'll buy you a beer when I'm in Tokyo next time!

nama-gatsuo commented 3 years ago

Great! Apparently I'd missed description about Azure Kinect Body Tracking SDK in README... So I updated it now. If you had env variable AZUREKINECT_BODY_SDK which points to BT SDK root, then no need to copy dlls.

Let me close this issue. Thanks again both @120decibel and @YongsunLee for cooperation. Feel free to reopen when problems still exist.