search

baldassarreFe / ws-vrd

17 stars 1 forks source link

Can you please provide the config for reproducing HICO result? #6

Open Charles-Xie opened 3 years ago

Charles-Xie commented 3 years ago

Hi, thanks for sharing the code. It's great! I'm currently trying to reproduce the result. I followed the instructions in data/README.md to set up dataset and prepare graphs. Then I tried to train the model. As the command for training is not provided, I use: python3 -m xib.train config/train.yaml config/visual_relations_hico.yaml config/relational_network.yaml And all the 3 config files used here are provided in the repository and I did not modify them at all. I have noticed that some details are not the same with what you mentioned in your paper, like max_epochs, batch_size, model channel and layer numbers, but I'm not sure how important these details are. And currently the result for 35 epochs is:

test_d2/gpu_mb: 1093
test_d2/pc/hoi/mAP: 0.19032068802048485
test_d2/vr/hoi/mAP/all: 0.20952911755835715
test_d2/vr/hoi/mAP/nonrare: 0.22004295744810923
test_d2/vr/hoi/mAP/rare: 0.17433061010136106
test_d2/vr/phrase/recall_at_100: 0.6908222436904907
test_d2/vr/phrase/recall_at_50: 0.41690579056739807
test_d2/vr/relationship/recall_at_100: 0.5246353149414062
test_d2/vr/relationship/recall_at_50: 0.3488341271877289

As you can see, the result on HICO (mAP 20.95%) is not close to the one reported in the paper (28.25 without h-o relational prior). Can you give me more instructions, like sharing the config file that can reproduce the result?

youngfly11 commented 2 years ago

Hope for the response! It's 2022