real-time continuous webcam prediction

[styap94]

hi, may i know, instead of reading an original video file, extract features and sequences, and make prediction, is that possible to let the program read from webcam and predict in real-time continuously with different set of actions? for example, apply-lipstick, followed by apply eye make up.

[mesmerx]

you can grab n frames, put in a tensor, or make a temp file, predict this file, and then remove/restart the loop this way you predict in real time, i think

[styap94]

is it possible to reach real time? because we not only need to create frames while streaming, we need to extract the features and create the sequence.npy file for prediction. it could take quite sometime to extract sequence of features.

[mesmerx]

Train the model , and then with a loaded model uses opencv extract lhe first 300 frames and save in array, predict that array, and in next loop add one frame and remove one, so you can predict in real time

[mesmerx]

I believe this could run at 20 fps or more

[styap94]

I’m using lstm model. Yes, I used the opencv to extract frames. But then it takes some time to extract the features from the frames created(as the author mentioned, lstm will use the feature extracted for prediction).
I think the problem is the feature extraction part. Ps: I’m very new to tf and machine learning. Thanks for explaining to me so patiently.

[mesmerx]

No problem at all The frame calculations is fast for a small sample, for 300 frames is almost no hardproblem, i dos with 2 vídeos with 30 seg and goes without Dalay at Sol. But give a try, to know

[mesmerx]

when i finish my actual model i will ttry make that,if you can wait

[styap94]

It’s good to know you will try to make this. I appreciate that so much. Hope you can finish ur model soon. Haha

[styap94]

another question to ask, is that possible to deploy this keras trained model (hdf5) to android ?

[mesmerx]

I cant se why not, Just a tip, Save only weights and then save json. Its faster oi load

[mesmerx]

I made a custom demo script, i will push to my git

[styap94]

great!! i will go to have a try

[styap94]

@mesmerx i did not see the demo script that you mentioned.

[mesmerx]

I have not pushed yet, Just making some work

[styap94]

@mesmerx have you done it? i still unable to solve the problem

[Lelo123]

Dos anyone been able to do this?

[Lelo123]

@mesmerx and @hanako94

[styap94]

you can convert the keras .hdf5 to tensorflow .pb format and then plug to your tensorflow android project https://github.com/amir-abdi/keras_to_tensorflow

[Lelo123]

Thanks. Just one more question, i'm still very new to machine learning, but wasn't suppose to exist a Validation and test set? In this cases the Validation set seems to be the test set for me

[Lelo123]

@hanako94

[drewgash]

@mesmerx

when i finish my actual model i will ttry make that,if you can wait

Were you able to complete this?