search

UttaranB127 / STEP

Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits
https://gamma.umd.edu/step
MIT License
69 stars 12 forks source link

Some questions about the dataset #7

Closed zsh2000 closed 3 years ago

zsh2000 commented 3 years ago

Hi,

Thanks for the code and dataset you have offered! I find myself really interested in your work!

I have some questions about the dataset as follows:

  1. I want to visualize the motion sequences in the dataset, but I find that the root joint (joint #0) is not at the origin of the coordinates, which is different from the ELMD dataset setting. (visualization in the first figure below) Also, when I try relocating the root joint to the origin, the scales and visual angles of the gaits vary a lot. (visualization in the second figure below) I wonder whether I need to do some preliminary works before visualizing the gaits.

  2. The synthetic gaits in the dataset seem a little strange. After performing visualization, it looks like the person is dragging his/her legs rather than walking.

  3. I wonder how the emotion labels are annotated. Are they annotated by pretrained models, or annotated manually?

Many thanks and best wishes! Shuhong

没用 elmd
UttaranB127 commented 3 years ago

Hi Shuhong,

Many thanks for your interest in our work!

  1. We visualized the gaits in a Unity environment. I am sharing the Unity project with you here. In order to visualize, you have to create a new Unity project with the given Assets, attach the RGB.cs script to the GameObject, and run the Unity project. The RGB.cs script reads in the.h5 files of the gaits.
  2. Yes, the synthetic gaits are not the best. We improved on this in our future projects, especially in https://github.com/UttaranB127/GeneratingEmotiveGaits.
  3. The emotions were annotated manually by domain experts.

Best regards, Uttaran

zsh2000 commented 3 years ago

Hi,

Thanks so much for your quick reply!! I will try it!

In addition, I wonder whether the experts do the annotation on the raw skeleton data, or they do it after the skeletons are rendered to RGB videos (or the AR environment).

Best regards, Shuhong

UttaranB127 commented 3 years ago

They actually did it on the raw RGB videos of the participants, from which we later extracted the skeletons and rendered those skeletons in Unity. Unfortunately, we cannot share the raw RGB videos at this point due to confidentiality constraints.

zsh2000 commented 3 years ago

Thank you so much!! I get it!