SizheAn / mRI

Data repo for mRI: Multi-modal 3D Human Pose Estimation Dataset using mmWave, RGB-D, and Inertial Sensors
Creative Commons Zero v1.0 Universal
52 stars 8 forks source link

[Please help] Project reproductability on own data #3

Closed AbdellahUnaide closed 1 year ago

AbdellahUnaide commented 1 year ago

Hello,

I'm using the same material (IWR1443 + One kniect). I'm getting the data in the good format (5D from iwr and 3D from Kinect). I'm trying to reproduce your work on my own data (defining now poses). Could you please give me some instructions (or data preprocessing) to generate the npy files for each type of data (radar and RGB).

My other question is : You mentioned in the article that you group 3 frames of radar data but i see on the npy file that you have have only 3d (not 5d) 17 matrix in the data tensor.

PS : I can propose a better way to get 3d data from kinect in better way if you are open for people participation.

Thank you in advance for help.

SizheAn commented 1 year ago

Hi,

Thanks for the interest in our paper.

For radar HPE, you can refer to our previous work MARS and its paper, the only two difference is:

  1. In MARS, we use a single kinect SDK to generate the 3D human key point ground truth while in mRI we use two kinects' RGB via computer vision approach.
  2. In MARS, we only use each frame, so the data dimension is (8,8,5). In mRI we fuse three frames such that it becomes (14,14,5). We reshape the data such that each of the five channel is a dimension (x,y,z,Dopler,Intensity). For more information you can refer to our MARS paper

For your other question, is your better way using Kinect SDK? We tried that in MARS but it is not stable and noiser.

AbdellahUnaide commented 1 year ago

Hello, Thank you for the clear answer. I had also previously a look on MARS work. Sorry, for using the term 'better' i meant the ease of use sense and noise free implementation (comparing to the openKinect implementation). I was talking about an open source work based on vision also. Called 'kinectron server'...

Thank you again for this great work. Issue to be closed.