search

SensorsINI / DHP19

Repository for the Dynamic Vision Sensor 3D Human Pose Dataset (DHP19).
MIT License
42 stars 10 forks source link

MPJPE Table Mean Values #14

Open bald6354 opened 3 years ago

bald6354 commented 3 years ago

Can you please describe how mean 3D MPJPE was calculated in Table 5 of the paper? For example, row #1 has a mean of mean([89.48 134.38 70.31 123.27 146.67])=112.82, but a reported mean of 115.04. I calculate column #1 mean to be 74.78, but the reported mean is 59.79. There must be some weighting applied, but I cannot figure it out. It is based on sample length? Would it make sense to normalize to the length of the sample? Thanks!

image

gianscarpe commented 3 years ago

Up, I found the same issue on the reported table

pmorerio commented 3 years ago

Up, same here. Maybe simply a matter of different number of frames for each movement?

enrico-c commented 3 years ago

Hi, @bald6354, @gianscarpe, @pmorerio,

The reported average values are the averages of all the samples in each category, or essentially weighted averages. I believe a weighted average might be more informative than the average of averages, but I understand this was not clearly explained and that it is common practice to report arithmetic averages.

Also, note that when evaluating the CNN with the frames from the repository, there are some differences in the averages of single recordings compared to table 5. This is due to changes and fixes done before the release of the code. However, the averages over movements, sessions, subjects, and the global average (all weighted averages) are within a few mm from the averages in table 5