Training on custom dataset

[NomiMalik0207]

Hi Daniil, Nice work, I am using custom dateset, I have done all the preprocessing of data. When I started training, got these warnings [WARNING] Not found pre-trained parameters for initial_stage.heatmaps.1.0.weight [WARNING] Not found pre-trained parameters for initial_stage.heatmaps.1.0.bias [WARNING] Not found pre-trained parameters for initial_stage.pafs.1.0.weight [WARNING] Not found pre-trained parameters for initial_stage.pafs.1.0.bias [WARNING] Not found pre-trained parameters for refinement_stages.0.trunk.0.initial.0.weight [WARNING] Not found pre-trained parameters for refinement_stages.0.heatmaps.1.0.weight [WARNING] Not found pre-trained parameters for refinement_stages.0.heatmaps.1.0.bias [WARNING] Not found pre-trained parameters for refinement_stages.0.pafs.1.0.weight [WARNING] Not found pre-trained parameters for refinement_stages.0.pafs.1.0.bias What should be the cause? Thank you...

[Daniil-Osokin]

Hi! Looks like you have changed some layers in the network, so weights from the pre-trained model cannot be copied directly. Warnings says for which layers the weights cannot be copied. It is ok, if network topology is changed.

[Daniil-Osokin]

Hope, it is clear now.

[NomiMalik0207]

It is clear but the point is I didnt change the layers of network butr yes I changed the dataset. What about the final results, either it is going to change the final result of not?

[Daniil-Osokin]

I believe network layers were changed, you can check it with git diff command. If no, these warnings look strange if pre-trained model was used.

[NomiMalik0207]

Hi Daniil, I am training on custom dataset. I donot have info like bounding boxes, scale_provided. Can you please tell me how I can train without these type of attributes?

[Daniil-Osokin]

This is needed for crops around each person during training, check #171. Boxes can be obtained as bounding boxes around all keypoints of a single person, you can do crops around each such box.

[NomiMalik0207]

In transformation, I am not using scaling (have commented this line). My question is is it necessary to use these info for getting better results? I have trained a network without bbox etc. Got improved loss values but when tried to draw on images, the key points scattered on whole screen of image. You can see in attached image.

[NomiMalik0207]

When I trained for longer epochs, the network didnt draw anything on image. Can you explain the possible reasons behind this?

[Daniil-Osokin]

No, it is not necessary to use such augmentation get better results than this image. This image can be from not properly trained network, looks like a bug somewhere.

[NomiMalik0207]

I tried to overfit the network by taking 500 images and its annotations. The results are same. In my data, i just dont have obj_pos and bounding boxes. Other than this info, I have all the information of COCO format. But still model is suffering with the same problem. Can you suggest some changes?

[NomiMalik0207]

At the same time, I also have some queries

1. in prepare_train_labels.py you are putting (x,y, confidence) and confidence =2 for missing joints and confidence =1 for complete joint. Where these confidence values are usable?
2. Can you explain the values of sigmas according to joints?

[Daniil-Osokin]

Hi! Keypoint type is used during heatmap building, missing joints are omitted in heatmaps. For the sigmas check #106.

[NomiMalik0207]

Thank you for your reply. As I want to check the dataset by using a small set of images and want the network to overfit. for this check, do I need to train all the 3 steps of training mentioned in repo (mobilenet, weights only and stages)? If yes, how much iterations are required if I have 500 images?

[Daniil-Osokin]

To overfit, just overfit, No need to do multiple stages for this. However, I suggest to use MobileNet weights to start training from.

[NomiMalik0207]

I have tried to use mobileNet weights provided by you on google derive, but still is giving the below warnings for custom dataset. but it is working for COCO. [WARNING] Not found pre-trained parameters for initial_stage.heatmaps.1.0.weight [WARNING] Not found pre-trained parameters for initial_stage.heatmaps.1.0.bias [WARNING] Not found pre-trained parameters for initial_stage.pafs.1.0.weight [WARNING] Not found pre-trained parameters for initial_stage.pafs.1.0.bias [WARNING] Not found pre-trained parameters for refinement_stages.0.trunk.0.initial.0.weight [WARNING] Not found pre-trained parameters for refinement_stages.0.heatmaps.1.0.weight [WARNING] Not found pre-trained parameters for refinement_stages.0.heatmaps.1.0.bias [WARNING] Not found pre-trained parameters for refinement_stages.0.pafs.1.0.weight [WARNING] Not found pre-trained parameters for refinement_stages.0.pafs.1.0.bias which pretty awkward.

[Daniil-Osokin]

Dear Muhammad, the warnings just have informed you, that there are no pre-trained weights for the layers you have defined un the model. You can always check the code which prints it to understand the situation.

[NomiMalik0207]

I know, about the warning. The purpose of sharing here is to inform you about the the pre-trained weights you shared for mobilenet via google derive is not working for custom dataset. The question is why these weights are working for COCO, but not for MPII and other dataset having different number of keypoints?

[Daniil-Osokin]

No, they are working for layers, which have the same number of parameters that in the checkpoint (for them no warnings are shown).

[NomiMalik0207]

I think you didn't understand my question. I change this net = PoseEstimationWithMobileNet(num_refinement_stages) to net = PoseEstimationWithMobileNet(num_refinement_stages, num_heatmaps=15, num_pafs=30) after changing, the pretraining weights are not working. and without these weights, model is not learning properly.