search

KaihuaTang / Scene-Graph-Benchmark.pytorch

A new codebase for popular Scene Graph Generation methods (2020). Visualization & Scene Graph Extraction on custom images/datasets are provided. It's also a PyTorch implementation of paper “Unbiased Scene Graph Generation from Biased Training CVPR 2020”
MIT License
1.03k stars 228 forks source link

Fine-tuning on the pre-trained motif-sgdet #180

Closed bibekyess closed 1 year ago

bibekyess commented 1 year ago

❓ Questions and Help

Hello, Thanks for this awesome work! I want to fine-tune this pre-trained motif-sgdet model on another custom dataset. I also made the dataset of 1000 images following the guidelines as mentioned on previous issues. But, I didn't get an idea of how to fine-tune this on top of the already trained model? Can anyone who has already done this help me or give me some hint?

Thank you!

bibekyess commented 1 year ago

I think I solved it. For others with similar doubt, I randomly initialized the weight parameters for the following layers and used the same weight parameters as of the pretrained model for the other layers (in MotifPredictor): 'module.roi_heads.box.predictor.cls_score.weight', 'module.roi_heads.box.predictor.cls_score.bias', 'module.roi_heads.box.predictor.bbox_pred.weight', 'module.roi_heads.box.predictor.bbox_pred.bias', 'module.roi_heads.relation.predictor.context_layer.obj_embed1.weight', 'module.roi_heads.relation.predictor.context_layer.obj_embed2.weight', 'module.roi_heads.relation.predictor.context_layer.decoder_rnn.obj_embed.weight', 'module.roi_heads.relation.predictor.context_layer.decoder_rnn.out_obj.weight', 'module.roi_heads.relation.predictor.context_layer.decoder_rnn.out_obj.bias', 'module.roi_heads.relation.predictor.rel_compress.weight'], 'module.roi_heads.relation.predictor.rel_compress.bias', 'module.roi_heads.relation.predictor.freq_bias.obj_baseline.weight'.

narchitect commented 7 months ago

@bibekyess Hi, Thanks for sharing your solution. I tried your way to fine-tune their checkpoint of the SDGet model, but I can't find the weights you mentioned above in the log file. May I ask you to let me know your way in more detail? like how to find the weights you mentioned above, and how to initialize the weight parameters for fine-turning with a custom dataset. because I'm a beginner at using an SGG ML model..