Hi, the training results are lower than reported results.

[Dreamer312]

I git clone the code and start training on 2 A40 GPUs, 8 images per GPU. The hyperparameters remain the same with the original ones. Did I miss something? : (

System: pytorch 2.0.1 py3.10_cuda11.8_cudnn8.7.0_0 pytorch pytorch-cuda 11.8 h7e8668a_5 pytorch torchvision 0.15.2 py310_cu118 pytorch

`======================sgdet============================ R@20: 0.203254 R@50: 0.250673 R@100: 0.271782

relationship: above ======================sgdet============================ R@20: 0.028179 R@50: 0.047680 R@100: 0.063172

relationship: across ======================sgdet============================ R@20: 0.015873 R@50: 0.015873 R@100: 0.047619

relationship: against ======================sgdet============================ R@20: 0.008065 R@50: 0.008065 R@100: 0.016129

relationship: along ======================sgdet============================ R@20: 0.000000 R@50: 0.000000 R@100: 0.000000

relationship: and ======================sgdet============================ R@20: 0.005917 R@50: 0.005917 R@100: 0.017751

relationship: at ======================sgdet============================ R@20: 0.111198 R@50: 0.163596 R@100: 0.176330

relationship: attached to ======================sgdet============================ R@20: 0.001179 R@50: 0.001474 R@100: 0.006191

relationship: behind ======================sgdet============================ R@20: 0.111431 R@50: 0.186552 R@100: 0.226916

relationship: belonging to ======================sgdet============================ R@20: 0.000000 R@50: 0.000000 R@100: 0.000000

relationship: between ======================sgdet============================ R@20: 0.000000 R@50: 0.000000 R@100: 0.003472

relationship: carrying ======================sgdet============================ R@20: 0.126705 R@50: 0.159264 R@100: 0.183682

relationship: covered in ======================sgdet============================ R@20: 0.016071 R@50: 0.018452 R@100: 0.025595

relationship: covering ======================sgdet============================ R@20: 0.000000 R@50: 0.000000 R@100: 0.012903

relationship: eating ======================sgdet============================ R@20: 0.059603 R@50: 0.094923 R@100: 0.118102

relationship: flying in ======================sgdet============================ R@20: 0.000000 R@50: 0.060606 R@100: 0.060606

relationship: for ======================sgdet============================ R@20: 0.022814 R@50: 0.035109 R@100: 0.039208

relationship: from ======================sgdet============================ R@20: 0.000000 R@50: 0.000000 R@100: 0.000000

relationship: growing on ======================sgdet============================ R@20: 0.000000 R@50: 0.000000 R@100: 0.000000

relationship: hanging from ======================sgdet============================ R@20: 0.000000 R@50: 0.003984 R@100: 0.003984

relationship: has ======================sgdet============================ R@20: 0.270150 R@50: 0.329975 R@100: 0.352970

relationship: holding ======================sgdet============================ R@20: 0.213111 R@50: 0.251752 R@100: 0.267716

relationship: in ======================sgdet============================ R@20: 0.098441 R@50: 0.143542 R@100: 0.169576

relationship: in front of ======================sgdet============================ R@20: 0.030335 R@50: 0.049338 R@100: 0.064679

relationship: laying on ======================sgdet============================ R@20: 0.081081 R@50: 0.121622 R@100: 0.130631

relationship: looking at ======================sgdet============================ R@20: 0.029451 R@50: 0.041499 R@100: 0.059572

relationship: lying on ======================sgdet============================ R@20: 0.040816 R@50: 0.051020 R@100: 0.051020

relationship: made of ======================sgdet============================ R@20: 0.000000 R@50: 0.000000 R@100: 0.000000

relationship: mounted on ======================sgdet============================ R@20: 0.000000 R@50: 0.000000 R@100: 0.000000

relationship: near ======================sgdet============================ R@20: 0.088555 R@50: 0.148588 R@100: 0.188174

relationship: of ======================sgdet============================ R@20: 0.188986 R@50: 0.256768 R@100: 0.279926

relationship: on ======================sgdet============================ R@20: 0.222539 R@50: 0.277110 R@100: 0.302019

relationship: on back of ======================sgdet============================ R@20: 0.000000 R@50: 0.000000 R@100: 0.000000

relationship: over ======================sgdet============================ R@20: 0.023573 R@50: 0.034739 R@100: 0.042184

relationship: painted on ======================sgdet============================ R@20: 0.000000 R@50: 0.000000 R@100: 0.000000

relationship: parked on ======================sgdet============================ R@20: 0.048555 R@50: 0.084791 R@100: 0.108387

relationship: part of ======================sgdet============================ R@20: 0.000000 R@50: 0.000000 R@100: 0.000000

relationship: playing ======================sgdet============================ R@20: 0.000000 R@50: 0.000000 R@100: 0.090909

relationship: riding ======================sgdet============================ R@20: 0.215077 R@50: 0.267793 R@100: 0.282938

relationship: says ======================sgdet============================ R@20: 0.000000 R@50: 0.083333 R@100: 0.083333

relationship: sitting on ======================sgdet============================ R@20: 0.098548 R@50: 0.150509 R@100: 0.164636

relationship: standing on ======================sgdet============================ R@20: 0.048238 R@50: 0.071938 R@100: 0.090096

relationship: to ======================sgdet============================ R@20: 0.000000 R@50: 0.000000 R@100: 0.000000

relationship: under ======================sgdet============================ R@20: 0.074767 R@50: 0.101710 R@100: 0.121233

relationship: using ======================sgdet============================ R@20: 0.114583 R@50: 0.140625 R@100: 0.170000

relationship: walking in ======================sgdet============================ R@20: 0.000000 R@50: 0.004695 R@100: 0.004695

relationship: walking on ======================sgdet============================ R@20: 0.038874 R@50: 0.089455 R@100: 0.111126

relationship: watching ======================sgdet============================ R@20: 0.008418 R@50: 0.061785 R@100: 0.083550

relationship: wearing ======================sgdet============================ R@20: 0.386910 R@50: 0.412767 R@100: 0.421221

relationship: wears ======================sgdet============================ R@20: 0.004430 R@50: 0.024502 R@100: 0.044608

relationship: with ======================sgdet============================ R@20: 0.029754 R@50: 0.070640 R@100: 0.093761

======================sgdet mean recall with constraint============================ mR@20: 0.05724454045111795 mR@50: 0.08143982200281324 mR@100: 0.09561244052384507 Averaged stats: class_error: 60.00 sub_error: 50.00 obj_error: 0.00 rel_error: 75.00 loss: 16.0778 (18.9691) loss_ce: 0.2776 (0.4303) loss_bbox: 0.8823 (0.9988) loss_giou: 1.0329 (1.0426) loss_rel: 0.4454 (0.6207) loss_ce_0: 0.3065 (0.4637) loss_bbox_0: 1.0056 (1.1189) loss_giou_0: 1.1239 (1.1814) loss_rel_0: 0.4299 (0.5930) loss_ce_1: 0.3023 (0.4488) loss_bbox_1: 0.9873 (1.0519) loss_giou_1: 1.0969 (1.1049) loss_rel_1: 0.3992 (0.5942) loss_ce_2: 0.2839 (0.4382) loss_bbox_2: 0.8772 (1.0255) loss_giou_2: 1.0365 (1.0784) loss_rel_2: 0.4104 (0.5954) loss_ce_3: 0.2801 (0.4330) loss_bbox_3: 0.7734 (1.0091) loss_giou_3: 1.0344 (1.0574) loss_rel_3: 0.4219 (0.5989) loss_ce_4: 0.2719 (0.4310) loss_bbox_4: 0.7968 (0.9987) loss_giou_4: 1.0301 (1.0459) loss_rel_4: 0.4093 (0.6088) loss_ce_unscaled: 0.2776 (0.4303) class_error_unscaled: 28.5714 (34.1161) sub_error_unscaled: 33.3333 (54.1432) obj_error_unscaled: 33.3333 (48.1875) loss_bbox_unscaled: 0.1765 (0.1998) loss_giou_unscaled: 0.5165 (0.5213) cardinality_error_unscaled: 7.0000 (7.0240) loss_rel_unscaled: 0.4454 (0.6207) rel_error_unscaled: 56.2500 (66.5290) loss_ce_0_unscaled: 0.3065 (0.4637) loss_bbox_0_unscaled: 0.2011 (0.2238) loss_giou_0_unscaled: 0.5619 (0.5907) cardinality_error_0_unscaled: 7.0000 (8.3696) loss_rel_0_unscaled: 0.4299 (0.5930) loss_ce_1_unscaled: 0.3023 (0.4488) loss_bbox_1_unscaled: 0.1975 (0.2104) loss_giou_1_unscaled: 0.5484 (0.5524) cardinality_error_1_unscaled: 8.0000 (8.0064) loss_rel_1_unscaled: 0.3992 (0.5942) loss_ce_2_unscaled: 0.2839 (0.4382) loss_bbox_2_unscaled: 0.1754 (0.2051) loss_giou_2_unscaled: 0.5182 (0.5392) cardinality_error_2_unscaled: 5.0000 (7.8084) loss_rel_2_unscaled: 0.4104 (0.5954) loss_ce_3_unscaled: 0.2801 (0.4330) loss_bbox_3_unscaled: 0.1547 (0.2018) loss_giou_3_unscaled: 0.5172 (0.5287) cardinality_error_3_unscaled: 6.0000 (7.5166) loss_rel_3_unscaled: 0.4219 (0.5989) loss_ce_4_unscaled: 0.2719 (0.4310) loss_bbox_4_unscaled: 0.1594 (0.1997) loss_giou_4_unscaled: 0.5150 (0.5229) cardinality_error_4_unscaled: 5.0000 (6.8948) loss_rel_4_unscaled: 0.4093 (0.6088) Accumulating evaluation results... DONE (t=127.17s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.132 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.262 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.115 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.032 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.086 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.183 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.210 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.330 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.337 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.136 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.272 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.403`

[yrcong]

Hi,

I didn't upload the post-processing code so this seems the results of the old version:)

Someone asked me about this. Sorry these months I have no time to update this repo. You could use this draft function instead of the one in the repo. I will update the repo as soon as possible.

Btw, it is better to use multigpu with small batch size (e.g. 2) to train the model. In my experiments this can bring better results.

[Dreamer312]

Well, thanks alot! I am going to test with your code!

[48215]

我也遇到了相关问题，具体解决问题您能帮我解决一下吗，我的邮箱是1914065662@qq.com,十分感谢您的帮助

[AlphaGoooo]

Hi,

I didn't upload the post-processing code so this seems the results of the old version:)

Someone asked me about this. Sorry these months I have no time to update this repo. You could use this draft function instead of the one in the repo. I will update the repo as soon as possible.

Btw, it is better to use multigpu with small batch size (e.g. 2) to train the model. In my experiments this can bring better results.

@yrcong How to use the evaluate_rel_batch.py? Could you please give an instruction like readme or something like that, thanks.