JHMDB dataset generation/parsing

[cboillet]

Hello, I would like to understand how to read the .csv dataset files for the JHMDB dataset. What it the format? Lines, columns signification? How was those files created from the .mat joints positions from the original JHMDB dataset? Thank you very much in advance

[noboevbo]

I load the JHMDB record as an intermediate representation into a database and export the EHPI from there. I plan to publish the data manager with the DB intermediate representation, but unfortunately I don't have the time right now. I have entered the part for the JHMDB import as well as the EHPI export from the database in the following GIST: https://gist.github.com/noboevbo/76437fa4303e2256d08abbf601f2198c

The importer also contains code to create and save joints from our 2D pose recognition pipeline in addition to the ground truth. The code is not tidy, but maybe it will help you anyway. The imports nobos_dataset_manager.xyz are mainly data models, thus not really relevant for the actual import logic, nobos_commons is included in my published repositories.

I'm sorry that I can only help in this way at the moment, I hope that I'll soon be able to clean up the Dataset Manager and put it online.

[cboillet]

Thanks a lot I will see what I can do already from there. Would you already know when you would plan on releasing the Dataset Manager?

[cboillet]

Hello Noboevbo, Thanks for the gist it is useful for the import. The files x*.csv, y*.csv have the following type of content:

1,19
1,19
1,19
1,19

I imagine the first digit is the action class. What does the second digit represents? does the row in the file contains an information? Thank you very much in advance for your guidance

[noboevbo]

Yes, the first digit is the action class. The second index is the unique action, thus every line with this index corresponds to an the same action (in JHMDB == the same video). The row of y.csv just encodes to which row in x.csv the label corresponds to.

[cboillet]

Ok thank you I understand now, And the x_*.csv file is composed of the 15 joints keypoints as following: joint.score, joint.x and joint.y for one or more human in the action video. I guess you created a custom dictionary between the action index and original JHMDB videos?

[noboevbo]

Sorry for the delay, I uploaded the dataset manager to github. The JHMDB im- and export should work, other functions may be buggy.

Have a look at https://github.com/noboevbo/nobos_dataset_manager.

[TGyAlDeen]

That's really nice of you.. i'm gonna check it now. i just wonder how to connect real life videos or came.. i know that i need to make my front end.. for example openpose output... but in general is it possible with your code !! can you provide guidance

[noboevbo]

That's really nice of you.. i'm gonna check it now. i just wonder how to connect real life videos or came.. i know that i need to make my front end.. for example openpose output... but in general is it possible with your code !! can you provide guidance

I think this question is not directly related to this topic. If you want to use the EHPI Code with videos (webcam?) have a look at https://github.com/noboevbo/ehpi_action_recognition/blob/master/ehpi_action_recognition/run_ehpi.py. It uses an "input_provider" which .. provides input. By default it uses a connected webcam, theres also commented code on how to use an img directory (with content: img1.jpg, img2.jpg, ...). If you want to use videos extract the single frames as images and use the ImgDirProvider or you could write your own VideoProvider.

[TGyAlDeen]

yeah, the question might not be related to the topic. but thanks for your answer Thanks, it's really nice of you...