search

microsoft / scene_graph_benchmark

image scene graph generation benchmark
MIT License
382 stars 86 forks source link

KeyError: 'gt_labels' during training #46

Open yizhe-ang opened 2 years ago

yizhe-ang commented 2 years ago

When attempting to perform training using tools/train_sg_net.py and a config file like sgg_configs/vg_vrd/rel_danfeiX_FPN50_nm.yaml, I receive the following error:

Traceback (most recent call last):
  File "tools/train_sg_net.py", line 225, in <module>
    main()
  File "tools/train_sg_net.py", line 218, in main
    model = train(cfg, args.local_rank, args.distributed)
  File "tools/train_sg_net.py", line 110, in train
    meters
  File "/home/scene_graph_benchmark/maskrcnn_benchmark/engine/trainer.py", line 94, in do_train
    loss_dict = model(images, targets)
  File "/home/miniconda3/envs/sg_benchmark/lib/python3.7/site-packages/torch/nn/modules/module.py", line
727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/miniconda3/envs/sg_benchmark/lib/python3.7/site-packages/torch/nn/parallel/distributed.py",
 line 619, in forward
    output = self.module(*inputs[0], **kwargs[0])
  File "/home/miniconda3/envs/sg_benchmark/lib/python3.7/site-packages/torch/nn/modules/module.py", line
727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/scene_graph_benchmark/scene_graph_benchmark/scene_parser.py", line 319, in forward
    x_pairs, prediction_pairs, relation_losses = self.relation_head(features, predictions, targets)
  File "/home/miniconda3/envs/sg_benchmark/lib/python3.7/site-packages/torch/nn/modules/module.py", line
727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/scene_graph_benchmark/scene_graph_benchmark/relation_head/relation_head.py", line
211, in forward
    = self.rel_predictor(features, proposals, proposal_pairs)
  File "/home/miniconda3/envs/sg_benchmark/lib/python3.7/site-packages/torch/nn/modules/module.py", line
727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/scene_graph_benchmark/scene_graph_benchmark/relation_head/neural_motif/neuralmotif
.py", line 135, in forward
    = _get_tensor_from_boxlist(proposals, 'gt_labels')
  File "/home/scene_graph_benchmark/scene_graph_benchmark/relation_head/sparse_targets.py", line
 61, in _get_tensor_from_boxlist
    assert proposals[0].extra_fields[field] is not None
KeyError: 'gt_labels'

Has anyone else encountered the same problem?

tao123322 commented 2 years ago

I also encountered the same problem. Has your problem been solved?

uehara-mech commented 2 years ago

I've encountered the same problem. Is there any solution?

albertmundu commented 2 years ago

I checked the .extra_fields from the BoxList class instance proposals[0] and found out that there are no gt_labels key but labels; so I replaced gt_labels with labels wherever required and got it working.

Reference example

print(proposals[0].extra_fields)
{'boxes_all': tensor([[[575.3116, 493.3303, 651.8593, 599.0000],
         [572.1199, 475.3301, 669.7687, 599.0000],
         [576.1859, 476.3813, 664.0187, 599.0000],
         ...,
         [577.2058, 480.1399, 650.8989, 599.0000],
         [574.8374, 439.8929, 667.1219, 599.0000],
         [578.2539, 478.6823, 665.7329, 597.5700]],

        [[760.1421, 517.9285, 798.9736, 585.5206],
         [758.9906, 492.4811, 799.0000, 591.2639],
         [761.8644, 495.5226, 797.3476, 585.2558],
         ...,
         [760.3383, 512.7456, 797.5162, 584.3093],
         [762.9234, 460.8504, 798.2347, 591.5004],
         [762.5838, 496.3992, 797.5012, 584.9985]],

        [[217.4372, 464.8350, 344.7936, 580.2131],
         [209.3953, 464.6660, 353.4158, 581.5388],
         [219.9669, 473.1698, 345.0997, 575.7039],
         ...,
         [214.2564, 476.5181, 338.5954, 578.5875],
         [217.1255, 454.1913, 349.7122, 584.5389],
         [213.6511, 471.9595, 347.7159, 576.5269]],

        ...,

        [[756.3811, 443.4718, 798.9641, 599.0000],
         [758.5258, 442.2567, 799.0000, 599.0000],
         [761.8578, 443.7815, 798.0120, 593.9513],
         ...,
         [757.6052, 450.4004, 797.7355, 591.3800],
         [760.8566, 432.0017, 798.4744, 597.9771],
         [762.1493, 443.6580, 798.4015, 592.5682]],

        [[ 86.8686, 442.1726, 224.6898, 599.0000],
         [ 98.9256, 433.0786, 233.1567, 599.0000],
         [121.9205, 438.3643, 221.5093, 593.1830],
         ...,
         [ 95.4544, 443.6589, 223.8852, 593.7196],
         [132.1051, 406.5965, 223.5276, 599.0000],
         [116.6622, 433.9163, 224.9746, 596.5552]],

        [[471.3473, 440.4014, 503.2437, 517.2510],
         [467.5936, 442.9140, 500.3037, 512.3618],
         [469.0378, 445.5520, 497.6595, 511.7478],
         ...,
         [468.8318, 445.6987, 499.7173, 512.9701],
         [469.9324, 443.6682, 496.9584, 513.1393],
         [468.8994, 445.5413, 498.3699, 512.1750]]], device='cuda:0'), 'scores': tensor([0.5157, 0.3642, 0.2451, 0.9075, 0.8277, 0.7750, 0.5613, 0.5009, 0.4823,
        0.3578, 0.3238, 0.2665, 0.2648, 0.2083, 0.1882, 0.1689, 0.1671, 0.8605,
        0.6772, 0.1858, 0.2564, 0.6414, 0.4974, 0.3765, 0.3628, 0.2200, 0.3872,
        0.1611, 0.1812, 0.9297, 0.8040, 0.5249, 0.2752, 0.2637, 0.1736, 0.9385,
        0.3597, 0.7236, 0.1654, 0.3856, 0.3744, 0.1625, 0.5811, 0.5752, 0.3696,
        0.3203, 0.6994, 0.2586, 0.2935, 0.1932, 0.3824, 0.6788, 0.6554, 0.6392,
        0.3846, 0.3330, 0.2181, 0.1797, 0.2947, 0.2925, 0.2890, 0.2682, 0.2522,
        0.2285, 0.1908, 0.6331, 0.1873, 0.7487, 0.5475, 0.3037, 0.1608, 0.4250,
        0.4595, 0.4514, 0.1832, 0.4708, 0.1881, 0.8529, 0.3856, 0.2878, 0.7058,
        0.8880, 0.6372, 0.4598, 0.2347, 0.9099, 0.1710, 0.3703, 0.3116, 0.2419,
        0.2178, 0.2177, 0.1760, 0.1742, 0.1689, 0.1619, 0.7784, 0.6541, 0.2444,
        0.1610], device='cuda:0'), 'box_features': tensor([[-2.6431, -4.5446,  2.6508,  ...,  3.9423, -2.5540,  4.5842],
        [ 2.1688, -6.4551,  3.3112,  ...,  4.0004, -5.7602,  7.0488],
        [ 0.2193,  0.5712, -4.2975,  ...,  0.2456,  3.2538, -5.8120],
        ...,
        [-2.3281, -1.5482,  2.4908,  ...,  0.5979, -1.4153,  3.8372],
        [-0.0944, -0.7335, -1.7120,  ..., -4.1399,  0.9713,  2.7227],
        [ 2.4646,  1.3760,  2.4534,  ...,  1.5840,  5.4696,  2.5356]],
       device='cuda:0', grad_fn=<SplitWithSizesBackward>), 'labels': tensor([  4,   4,  19,  22,  22,  22,  22,  22,  22,  22,  22,  22,  22,  22,
         22,  22,  22,  23,  23,  23,  25,  26,  26,  26,  26,  26,  31,  31,
         44,  47,  47,  47,  47,  47,  47,  57,  57,  60,  60,  61,  61,  61,
         62,  66,  66,  66,  67,  71,  73,  75,  76,  78,  78,  78,  78,  78,
         78,  80,  91,  91,  91,  91,  91,  91,  91,  96,  96,  99,  99,  99,
         99, 101, 111, 111, 111, 114, 114, 115, 115, 115, 124, 136, 136, 136,
        137, 139, 139, 145, 145, 145, 145, 145, 145, 145, 145, 145, 149, 149,
        149, 149], device='cuda:0'), 'scores_all': tensor([[1.2894e-01, 1.6794e-07, 7.2910e-05,  ..., 1.6956e-05, 5.3077e-02,
         1.8925e-07],
        [5.5617e-01, 1.6200e-07, 7.4476e-06,  ..., 5.8444e-06, 1.8381e-02,
         1.0729e-07],
        [2.4692e-01, 1.6499e-06, 7.4386e-06,  ..., 1.4503e-05, 4.3047e-05,
         4.9935e-06],
        ...,
        [7.4947e-02, 1.2510e-07, 5.3943e-06,  ..., 2.5121e-07, 6.5408e-01,
         2.3897e-07],
        [1.2687e-01, 1.2644e-05, 1.6872e-05,  ..., 8.0380e-05, 2.4440e-01,
         6.5623e-06],
        [3.6234e-01, 1.5064e-06, 1.1199e-05,  ..., 2.1323e-06, 1.6104e-01,
         9.1941e-06]], device='cuda:0')}