hysts
commented
2 years ago Hi, @GrayNiwako
Sorry for the confusion. What I meant to say in the README was that the images in the README demo were generated by the TADNE model, not that the training data were also generated. The training images were collected from the internet, because I thought the variations of pose and appearance of generated images might be limited.
As for the number of landmark annotations, I manually annotated 300 images and split them into 200-100 for training and validation data, making sure to include enough pose and appearance variations in the validation data. The number of annotations is very small, but it still took me days to annotate. About image selection, it is important to note that I didn't randomly choose images to annotate, but I manually selected relatively difficult images in terms of head pose, illumination, occlusion, etc., because it'd be a waste of time to annotate easy examples. Also, the annotation data were gradually increased in a human-in-the-loop cycle, where I annotated some images, trained models, and then manually checked the predictions to select the next images to annotate. I also tried semi-automatically selecting difficult images by checking consistency of predictions of multiple models and of rotated images.
FYI, the current model is still not good at dealing with occlusions and face with large deformations, etc., but it was good enough for what I wanted to do with this model, so I stopped there. I don't think the current number of training data is enough and I think it's easy to improve the model performance by increasing training data. I do not plan to release the training data due to copyright issues, but I think pseudo-labels by the current model is good enough to achieve the same or possibly better performance than the current model.
GrayNiwako
Thanks for your work! It’s very interesting!! May I ask you some questions? Did you manually annotate landmarks for the images generated by the TADNE model? And how many images does your training dataset include?