search

davisking / dlib-models

Trained model files for dlib example programs.
Creative Commons Zero v1.0 Universal
1.47k stars 375 forks source link

body pose model? #38

Open fm64hylian opened 4 years ago

fm64hylian commented 4 years ago

Hi, maybe this does not go here but is it possible to have something similar to 68 face landmarks but with full body or upper body? I am developing on unity and face detection is pretty much covered, however for body pose(my target is android) I haven't had any luck. I have tried ARcore(not compatible) AR foundation, OpenCV python, barracuda using an onnx model (not compatible) tried openpose, posenet, etc but I haven't found any pretrained body model similar to faces yet.

In this asset: https://assetstore.unity.com/packages/tools/integration/opencv-plus-unity-85928

there is an example that use face recognition, and by using the face and eye haar cascades plus the 68 face annotation landmarks it creates a real time recognition example. I want to replicate this for a full body, or upper body using body annotations with the fullbody and upper body haar cascades, is it possible? my goal is to create a fitting room using AR for android

sorry if it's not okay to post it here, I've been researching about it for 2 weeks and it's driving me crazy. it doesn't have to be that many landmarks, I'm ok with 13 or something

thank you

arrufat commented 4 years ago

I was also facing a similar problem, I tried to train a shape predictor for pose estimation, but I couldn't make it work either, especially for weird poses (it turns out some humans are really flexible)

Dlib has this loss layer: loss_mean_squared_per_channel_and_pixel, which is what they use in this approach to estimate landmark heatmaps: Simple Baselines for Human Pose Estimation and Tracking. When used with a ResNet50 backbone and an input size of 224x224, you can easily get inference times below 10 ms on a GPU, however, I don't think that would be suitable for an Android app, where there's no CUDA...

fm64hylian commented 4 years ago

Thank for your reply, it is indeed difficult without cuda(I am using windows with an Intel chip which makes it even worse) I haven't dig too much in models but I tried to run some like the body25 from open pose, I am completely new to deep learning so I kind of need a lead, is it possible to create a rough body pose model within one week? what would you recommend? I know that unity supports nn files as well, should I dig on tensorflow maybe? (they gave me 3 weeks to make this app lol)

arrufat commented 4 years ago

I am afraid I am not in a position to give you advice on how you should do your work. However, I will say that, given the amount of time you have left, you'd be better off using some already trained solutions like this one: https://github.com/edvardHua/PoseEstimationForMobile

fm64hylian commented 4 years ago

thanks so much! I will definetely give it a look. Another super noob question, the steps for training the model are there but, is it there like a generated file that can just drag to read it with opencv or similar? or do I have to compile something to generate that? I still don't understand how most pretrained models work