search

Nicholasli1995 / EvoSkeleton

Official project website for the CVPR 2020 paper (Oral Presentation) "Cascaded deep monocular 3D human pose estimation wth evolutionary training data"
https://arxiv.org/abs/2006.07778
MIT License
333 stars 44 forks source link

evolved dataset projection issue #21

Closed we1314love closed 3 years ago

we1314love commented 3 years ago

Hi, I used your evolved program to evolved my own dataset with the default parameters, my own datset has 161 samples, and the evolved result is 419, but after I project the evolved result(419) to 2D and check its xy position, the valid(if min(xy) <0, invalid) 2D pose is only 47.

My dataset(world coordinate) is not in the meter unit like Human36M, does this affect my evolved result?

My dataset only has 17 joints, the other joints I set the value with 1, does this affect my evolved result?

In my understanding you only check the valid of 3D pose(world coordinate), and use all the evolved data to train, can this evolved strategy guarantee your 2D projection is valid?

Thank you very much

Nicholasli1995 commented 3 years ago

Hi, I used your evolved program to evolved my own dataset with the default parameters, my own datset has 161 samples, and the evolved result is 419, but after I project the evolved result(419) to 2D and check its xy position, the valid(if min(xy) <0, invalid) 2D pose is only 47.

My dataset(world coordinate) is not in the meter unit like Human36M, does this affect my evolved result?

My dataset only has 17 joints, the other joints I set the value with 1, does this affect my evolved result?

In my understanding you only check the valid of 3D pose(world coordinate), and use all the evolved data to train, can this evolved strategy guarantee your 2D projection is valid?

Thank you very much

Hi, the location of the projected 2D joints depend on the 3D skeletons as well as the camera parameters. The default camera parameters is taken from H36M and should not produce subjects that lie outside of the image plane. If the projected keypoints lie outside of the image plane, the reason may be:

  1. You modified the skeleton size and the projected skeleton is too huge.
  2. You modified the location of the 3D skeleton.
  3. You modified the position of the camera.

"My dataset(world coordinate) is not in the meter unit like Human36M, does this affect my evolved result?" Reply: Certain operator such as crossover only modifies limb orientation and using it is not influenced by the skeleton size. However, the camera projection is size-dependent. Your projected skeletons can be too huge if your input 3D skeletons does not use the same unit.

"My dataset only has 17 joints, the other joints I set the value with 1, does this affect my evolved result?" Reply: This should not influence the result. Only 17 joints are used in this repository.

"In my understanding you only check the valid of 3D pose(world coordinate), and use all the evolved data to train, can this evolved strategy guarantee your 2D projection is valid?" Reply: The global location of the skeletons are taken from H36M and are not modified during evolution. By default the projected skeletons should lie inside the image plane. In fact, the location of skeletons and cameras can be controlled by specifiying the root position and camera translation.

we1314love commented 3 years ago

Hi, the location of the projected 2D joints depend on the 3D skeletons as well as the camera parameters.

Sorry to bother you with not explicitly illustration that I project my evolved 3D pose with my own corresponding parameter, I use the default parameter of your evolve.py to generate 3D pose.

Nicholasli1995 commented 3 years ago

Hi, the location of the projected 2D joints depend on the 3D skeletons as well as the camera parameters.

Sorry to bother you with not explicitly illustration that I project my evolved 3D pose with my own corresponding parameter, I use the default parameter of your evolve.py to generate 3D pose.

Hi, are the projected joints valid before evolution? Did you disable "modify bone length" option during the evolution? If no please try setting "modifying bone length" to False since that option may change the size of the subjects. You can disable it by setting opt.MBL to False.

we1314love commented 3 years ago

Hi, are the projected joints valid before evolution?

yes, I did the same 2D projection validity check on the 3D pose of my data before evolution to guarantee the validity of 3D pose, the result showed all the 2D projection is valid

Did you disable "modify bone length" option during the evolution? If no please try setting "modifying bone length" to False since that option may change the size of the subjects. You can disable it by setting opt.MBL to False.

Hi, Thank you, I found this issue and set the opt.MBL to False, this item did make some of 2D projection of my evolved pose invalid, besides this opt.MBL item, in exploration function, I also add one operation to check the valid of 2D pose, based on this two operation, all of the 2D projection of the evolved data is valid.

but I think the "modify bone length" option will enhance the variety of evolved pose, so I will further think how to modify this function to take effect on my data.

thank you very much for your reply

Nicholasli1995 commented 3 years ago

Closing inactive issues that already have answers. Re-open if needed.