Error when run with visual genome dataset

[978749951]

Hi GriffinLiang: When using vg dataset, the project works perfect. However, i have a problem when training it using visual genome dataset. More specifically, it works well when training while an "assert error" occurs when validating. @GriffinLiang @GoodDeeds could give me any advice?

The error trace is as follows: TRAIN:73000, Total LOSS:0.289731, Time:00:00:28 TRAIN:73500, Total LOSS:0.285681, Time:00:00:31 CLS TEST r50:76.310760, r50_zs:17.201166, r100:81.321226, r100_zs:28.613078 TEST Time:00:21:08 Traceback (most recent call last): File "train.py", line 120, in res.append((epoch,) + test_pre_net(net, args)+test_relnet(net, args)) File "/home/chenjinchang/Desktop/multi-modal/vrd-dsr-master/tools/../lib/model.py", line 112, in test_rel_net test_data = test_datalayer.forward() File "/home/chenjinchang/Desktop/multi-modal/vrd-dsr-master/tools/../lib/data_layers/vrd_data_layer.py", line 58, in forward return self.forwarddet() File "/home/chenjinchang/Desktop/multi-modal/vrd-dsr-master/tools/../lib/data_layers/vrd_datalayer.py", line 190, in forward det self.getDualMask(ih, iw, oBBox)] File "/home/chenjinchang/Desktop/multi-modal/vrd-dsr-master/tools/../lib/data_layers/vrd_data_layer.py", line 273, in _getDual Mask assert(mask.sum() == (y2 - y1) * (x2 - x1)) AssertionError

[GriffinLiang]

I think you may print the values of x1, x2, y1, and y2, since x2(y2) should be greater than x1(y1).

[978749951]

@GriffinLiang Thanks I have solved the problem with your help. I noticed that you provide a pre-trained faster-rcnn model named "faster_rcnn_1_20_7559.pth" which is corresponding to the vg dataset. Is there any pre-trained model for visual genome dataset? If not, how could I trian such a model. I try to use the pre-trained faster-rcnn model provided by Jianwei Yang, but pre-trained model's parameter size do not match to the setting of "detector.py".

[GriffinLiang]

Currently, there is no pre-trained model for visual genome dataset. You have to train the model using the Faster R-CNN code provided by Jianwei Yang. Since the version of the provided visual genome dataset is different from the version used by Jianwei Yang, the parameter sizes do not match.

[978749951]

Hi @GriffinLiang: I pre-trained a Faster R-CNN model and load it successful. However, I find that my pre-trained model is still not suitable for the data you provided. I notice that the reason why the problem occurs is as follows: I use version 1.2 of visual genome dataset and extract the most common 200 objects and the most common 100 relationships to build object and relationship vocab, respectively. But, I notice that my object vocab and relationship vocab is different from the vocab you provide. could you give me advice on how to construct the same object and relationship vocab as you? In this way, I could reuse the "so_prior.pkl", "train.pkl" , "test.pkl" and so on, which will save me a lot of time.

[GriffinLiang]

The script is not included in this project. Sorry about that.

[978749951]

Thank you all the same!

[Anakindai]

your help.

Could you tell how to solve this problem？ I meet the same problem.