I followed the tutorial, but keypoints didn't show

k0hh2 commented 3 years ago

Problem

Git clone Barracuda-PoseNet-Tutorial & I followed the tutorial, but keypoints didn't show There are no errors on the console. When played, it appears in the hierarchy window but not in the game view

I'm not good at English, so I'm sorry for the incomprehensible sentences Please tell me the solution.

What I tried

Changed from SetActive (show) to SetActive (true) to activate the object. Activated but not visible in Game View (Lines 114-116 of PoseSkeleton.cs)

for (int i= 0; i < jointPairs.Length; i++)
        {
            lines[i].SetActive(true);
            keypoints[jointPairs[i].Item1].gameObject.SetActive(true);
            keypoints[jointPairs[i].Item2].gameObject.SetActive(true);
        }

Execution video

https://gyazo.com/91dc4bf28bc64114af596c41ee71ec74 https://gyazo.com/fe9e6b364f5c0c26ed1ad890027240b3

My repository

https://github.com/k0hh2/Posenet-Barracuda

cj-mills commented 3 years ago

Hi @k0hh2, From the screen captures you shared, it appears that you are on Mac and are using the Compute Precompiled GPU backend. Since the key point GameObjects are not toggling on and off in the Hierarchy tab while the video plays, that would suggest that the model is either not receiving proper input or the inference engine is not working properly.

If you check the Stats in the Game View, does the framerate suggest that the model is actually running?

Even though Unity indicates that GPU inference is supported on Metal, could you try using the C Sharp Burst CPU backend?

I do not know which Mac you have and do not have any Macs to test the project myself. However, I have heard there are some issues with using the video player on AMD GPUs. Have you tried using the webcam as input?

It is also possible that there are bugs in the preview version of Barracuda used in this tutorial that affects performance on Macs. You could try downgrading the Barracuda package to version 1.3.0 and see if that helps.

Also, I get notified for comments on the blog so you don't need to repost.

k0hh2 commented 3 years ago

thank you for your answer. I'm using a MacBook Pro (13-inch, 2018) with a Big Sur 11.5.1 OS.

When I set it to cpu, keypoints were displayed. Sorry for the lack of confirmation. It didn't show up on gpu and webcam. The webcam is not displayed correctly so I will try to find a solution.

https://gyazo.com/f3fbd8015ce710248d7c3df1ef19b53a

cj-mills commented 3 years ago

@k0hh2 I believe a reader encountered this issue with the webcam on Mac when following my original tutorial. He was able to resolve the issue by following these steps. Hopefully it works for you as well.

"One hiccup I ran into on Step 7 was that my webcam's size wasn't immediately available. It defaulted to an ugly 16 x 16 image, which wasn't much useful.

Using the solution from this post - https://bit.ly/3oxk1xg - if you just turn on the webcam and wait, the dimensions will update with the correct number after a moment. So I:

Moved your whole start logic to a coroutine added import System.Collections at the top Just before videoHeight and videoWidth are set, added "yield return new WaitUntil(() => webcamTexture.height > 100);"

and everything else fell in line.”

I copied his posted solution from the comments section in this post. https://christianjmills.com/Barracuda-PoseNet-Tutorial-7/

Also, if you need to use the CPU backend, make sure to use the MobileNet model to get decent frame rates.

k0hh2 commented 3 years ago

Thank you for politely teaching me. I got an error when I moved the start method to a coroutine. This content is difficult for me and I can't seem to solve it on my own, so I'll try other methods that don't use posenet.

If you've tried something other than posenet and it doesn't work, I may ask again. At that time, thank you. I'm really thankful to you.

cj-mills / Barracuda-PoseNet-Tutorial