Issues with pre-trained Faster-RCNN used to train Motif's SGDet

[Namgyu97]

We attempted to SGDet train for Motifs using the pre-trained FasterRCNN provided by Motifs. However, although the model usually operates in the detector_test process, it is ineffective in the scene-graph learning process. Why is that??

We use RTX4090ti 1 GPUs, cuda 12.1. Below are the log files and yaml files for the files we used.

FasterRCNN command : python ./tools/detector_pretest_net.py --config ./configs/detector_R_101_FPN_1x.yaml

log.txt config.txt

Motifs : SGDet command: python ./tools/relation_train.net.py --config ./configs/VG150/SGG_FasterRCNN.yaml

config.txt log.txt

[Maelic]

Hi, it's been a while that I haven't trained a motifs model. From what I can see in your config file, your issue may be due to your low batch size (the variable IMS_PER_BATCH) which is 4. I usually trained all models with batch size 16 so I don't have to change the learning rate and steps factors. If you want to decrease the batch size (which I don't think you need with a 4090 GPU, but maybe you have memory issues) you should adjust the learning rate accordingly, a rule of thumb is to decrease the learning rate from the same proportion, which means in your case BASE_LR should be 0.01 / 4 = 0.0025

[Maelic]

Ok I think I found the issue with the motifs model, it was coming from the object features that were wrongly resized in the LSTM context encoder, I have fixed it. It should work now if you update the code and re-install (a clean pip install . should be enough).

However, I can't guarantee that you would be able to reproduce the original results from authors as a lot of things have changed in the codebase since then!

[Namgyu97]

Thank you. In the case of other models, it is running without problems. I will try installing the latest version.