search

stuarteiffert / RNN-for-Human-Activity-Recognition-using-2D-Pose-Input

Activity Recognition from 2D pose using an LSTM RNN
291 stars 77 forks source link

How to predict the label of keypoints using the lstm model #12

Closed kli017 closed 5 years ago

kli017 commented 5 years ago

Hello, Thanks for sharing your awesome project. I was wondering how to use this model to predict the label of some new keypoints? Could you provide a simple example? Thanks

seiffertacfr commented 5 years ago

Hi, Unsure what you mean exactly. Are you asking how to run inference using this model and a new sequence of poses? If so, please see issue #2 Online Inference

You're question sounds more like you are wanting to predict the dimension of the input data (ie whether a keypoint is an 'elbow' or a 'hand'), in which case using this model would be the wrong approach, but you would probably be able to do so by having a label for each dimension, and randomising the dimension order at input.

kli017 commented 5 years ago

@seiffertacfr Thanks for the help. Yes I was exactly asking how to run inference using this model and a new sequence of poses. I saved the model according to your answer in issue #2 and then try to restore it. However, it always failed to restore the model.ckpt. I am new to tensorflow, I was wondering if you can show me a more detailed example.. Thank you so much.

kli017 commented 5 years ago

By the way, when you want run online inference. Sometimes for some reason the detected keypoints of each person would less than 18, so the dimension of input would changed. How do you deal with this problem?

stuarteiffert commented 5 years ago

You need to make sure the dimension stays the same size. I simply included undetected keypoints as [0,0].

Whilst this is still a valid input, the network seemed to learn to distinguish these padded inputs from actual inputs, however a better approach would be to then not calculate the loss for the these inputs. I don't think I did this.