different joints

[ttdd11]

Very cool work. I'm wondering if I could apply this to a different data set. I have different 2D joints defined than the COCO dataset (some are the same) for my studies. How would I change the 3D data needed for training to accommodate for these?

[mks0601]

Hi,

For the new 2D dataset, you can follow data/MSCOCO/MSCOCO.py. For the different joint annotation, you can just change self.joint_name. Note that for the same joints with the 2D data, the joint name should be the same.

[mks0601]

By the way, RootNet does not estimate body joint coordinates. Maybe you are interested in PoseNet? In the PoseNet repo, 106th line of data/dataset.py, transform_joint_to_other_db handles different joint names.

[ttdd11]

Hey thanks for the reply. I was looking at both repos, sorry about that.

I'm not as worried about the joint names as how the 3D data will correspond to the 2D joints. Our lab has annotated data for other key points on the body (just a different set). What do I need to do for this 3Dpose net to work with my joints besides changing the dataset.py joint names? Since they do not meet the definition for the joints in mpii or coco, so that the end result is that the training yields our labs joints.

[mks0601]

Actually, except for dataset.py, you do not need to change anything. I separated each component in corresponding .py.

[ttdd11]

Okay just to re-iterate:

1. I download all the data, COCO, MPII, 3D data
2. Add my data in COCO format to these data as you describe
3. Modify the dataset.py to ensure the names match
4. When I train change the joint_num to correspond to the number of joints I have?

What I don't understand is how to ensure that the output corresponds to my joints, and not other joints present in the 2D datasets. Or do I just use my 2D data set along with one of the 3D datasets, and the the output corresponds to the joints in my 2D dataset, but calculated in 3D? Thanks for your help.

[mks0601]

Let's look at PoseNet repo At 106th line of common/base.py, self.joint_num is defined as joint_num of the 0th trainset_loader. The trainset_loader follows the order of the 13th line of main/config.py. For example, when you set trainset = ['Human3.6M', 'MSCOCO', 'MPII'], the joint_num is joint_num of the Humman3.6M dataset.

[mks0601]

To summarize, 0th trainset in main/config.py is reference dataset, and all other datasets follow that dataset joint_name.

[ttdd11]

Okay sounds good. I will give this a try with my 2D data being the 0th in the train set, and make sure that common names between that and the 3D data match.

[ttdd11]

And just to confirm, this should still work in the repo even if all the 2D joints are not defined in the 3D dataset?

[mks0601]

It does not matter. As I said, if you set 0th trainset as your dataset, there gonna be no problem.

[ttdd11]

Sounds good, I will give this a try!