`UnboundLocalError: local variable 'keep_tri' referenced before assignment`

[ShunchiZhang]

https://github.com/Jingkang50/OpenPSG/blob/bdd3c548257b99468e7adddfac7f1fd41a4321a7/configs/psgtr/psgtr_r50_psg.py#L108

I changed the above line into use_mask=False to try to only get the bounding box predictions rather than panoptic masks, but got this error local variable 'keep_tri' referenced before assignment on the following line at inference time after training the first epoch.

https://github.com/Jingkang50/OpenPSG/blob/bdd3c548257b99468e7adddfac7f1fd41a4321a7/openpsg/models/relation_heads/psgtr_head.py#L1171

I noticed keep_tri is defined on line 992, only when use_mask==True.

https://github.com/Jingkang50/OpenPSG/blob/bdd3c548257b99468e7adddfac7f1fd41a4321a7/openpsg/models/relation_heads/psgtr_head.py#L992

Could you kindly help me to resolve this issue or give any suggestions? Thank you so much :)

[Click Here for Full Error Log]

```python Traceback (most recent call last): File "tools/train.py", line 196, in main() File "tools/train.py", line 192, in main meta=meta) File "/home/shunchiz/mmdet/mmdet/apis/train.py", line 209, in train_detector runner.run(data_loaders, cfg.workflow) File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run epoch_runner(data_loaders[i], **kwargs) File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 54, in train self.call_hook('after_train_epoch') File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/mmcv/runner/base_runner.py", line 309, in call_hook getattr(hook, fn_name)(self) File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/mmcv/runner/hooks/evaluation.py", line 267, in after_train_epoch self._do_evaluate(runner) File "/home/shunchiz/mmdet/mmdet/core/evaluation/eval_hooks.py", line 119, in _do_evaluate gpu_collect=self.gpu_collect) File "/home/shunchiz/mmdet/mmdet/apis/test.py", line 98, in multi_gpu_test result = model(return_loss=False, rescale=True, **data) File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 886, in forward output = self.module(*inputs[0], **kwargs[0]) File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 110, in new_func return old_func(*args, **kwargs) File "/home/shunchiz/mmdet/mmdet/models/detectors/base.py", line 174, in forward Traceback (most recent call last): return self.forward_test(img, img_metas, **kwargs) File "/home/shunchiz/mmdet/mmdet/models/detectors/base.py", line 147, in forward_test return self.simple_test(imgs[0], img_metas[0], **kwargs) File "/home/shunchiz/OpenPSG/openpsg/models/frameworks/psgtr.py", line 144, in simple_test rescale=rescale) File "/home/shunchiz/mmdet/mmdet/models/dense_heads/base_dense_head.py", line 360, in simple_test return self.simple_test_bboxes(feats, img_metas, rescale=rescale) File "/home/shunchiz/OpenPSG/openpsg/models/relation_heads/psgtr_head.py", line 1187, in simple_test_bboxes File "tools/train.py", line 196, in main() File "tools/train.py", line 192, in main meta=meta) File "/home/shunchiz/mmdet/mmdet/apis/train.py", line 209, in train_detector runner.run(data_loaders, cfg.workflow) results_list = self.get_bboxes(*outs, img_metas, rescale=rescale) File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 198, in new_func File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run epoch_runner(data_loaders[i], **kwargs) return old_func(*args, **kwargs) File "/home/shunchiz/OpenPSG/openpsg/models/relation_heads/psgtr_head.py", line 899, in get_bboxes File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 54, in train self.call_hook('after_train_epoch') File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/mmcv/runner/base_runner.py", line 309, in call_hook getattr(hook, fn_name)(self) scale_factor, rescale) File "/home/shunchiz/OpenPSG/openpsg/models/relation_heads/psgtr_head.py", line 1171, in _get_bboxes_single File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/mmcv/runner/hooks/evaluation.py", line 267, in after_train_epoch self._do_evaluate(runner) File "/home/shunchiz/mmdet/mmdet/core/evaluation/eval_hooks.py", line 119, in _do_evaluate gpu_collect=self.gpu_collect) det_bboxes = torch.cat((s_det_bboxes[keep_tri], o_det_bboxes[keep_tri]), 0) UnboundLocalError: local variable 'keep_tri' referenced before assignment File "/home/shunchiz/mmdet/mmdet/apis/test.py", line 98, in multi_gpu_test result = model(return_loss=False, rescale=True, **data) File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 886, in forward output = self.module(*inputs[0], **kwargs[0]) File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 110, in new_func return old_func(*args, **kwargs) File "/home/shunchiz/mmdet/mmdet/models/detectors/base.py", line 174, in forward return self.forward_test(img, img_metas, **kwargs) File "/home/shunchiz/mmdet/mmdet/models/detectors/base.py", line 147, in forward_test return self.simple_test(imgs[0], img_metas[0], **kwargs) File "/home/shunchiz/OpenPSG/openpsg/models/frameworks/psgtr.py", line 144, in simple_test rescale=rescale) File "/home/shunchiz/mmdet/mmdet/models/dense_heads/base_dense_head.py", line 360, in simple_test return self.simple_test_bboxes(feats, img_metas, rescale=rescale) File "/home/shunchiz/OpenPSG/openpsg/models/relation_heads/psgtr_head.py", line 1187, in simple_test_bboxes results_list = self.get_bboxes(*outs, img_metas, rescale=rescale) File "/home/shunchiz/miniconda3/envs/mmdet/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 198, in new_func return old_func(*args, **kwargs) File "/home/shunchiz/OpenPSG/openpsg/models/relation_heads/psgtr_head.py", line 899, in get_bboxes scale_factor, rescale) File "/home/shunchiz/OpenPSG/openpsg/models/relation_heads/psgtr_head.py", line 1171, in _get_bboxes_single det_bboxes = torch.cat((s_det_bboxes[keep_tri], o_det_bboxes[keep_tri]), 0) UnboundLocalError: local variable 'keep_tri' referenced before assignment ```

[GSeanCDAT]

Hi Shunchi, thanks for your question! We will fix this bug as soon as possible.

BTW, if you want to see the bounding box triplet results, you can still set use_mask=True and switch eval_pan_rels=True here https://github.com/Jingkang50/OpenPSG/blob/bdd3c548257b99468e7adddfac7f1fd41a4321a7/openpsg/models/frameworks/psgtr.py#L16

to eval_pan_rels=False.

Under this condition, the model also outputs the bounding box triplets(but deduplicated ones, compared with setting use_mask=True).

[ShunchiZhang]

Hi Shunchi, thanks for your question! We will fix this bug as soon as possible.

BTW, if you want to see the bounding box triplet results, you can still set use_mask=True and switch eval_pan_rels=True here

https://github.com/Jingkang50/OpenPSG/blob/bdd3c548257b99468e7adddfac7f1fd41a4321a7/openpsg/models/frameworks/psgtr.py#L16

to eval_pan_rels=False.

Under this condition, the model also outputs the bounding box triplets(but deduplicated ones, compared with setting use_mask=True).

Hi Dr. Guo @GSeanCDAT, thanks for your kind reply!

I'm kind of confused here: will I get the non-deduplicated or deduplicated results with use_mask=True and eval_pan_rels=False?

Thank you so much!

[ShunchiZhang]

UPD: This will yield evaluation results that all numericals are zero

I guess it is because the model doesn't predict panoptic heads at all, since use_mask is False when training.

---

Also, I have additional 2 questions as follows:

If I'd only like to get the bboxes instead of panoptic masks, would use_mask=True

• decelerates the training process?
• makes the objective focusing on panoptic predictions rather than bbox prediction?

If the above 2 questions are both true, is it possible and correct to

1. Train with use_mask=False
2. Evaluate the checkpoint with use_mask=True and eval_pan_rels=False

as a workaround?

[GSeanCDAT]

Hi Shunchi, Here are my answers to your questions:

1. One-stage models with use_mask=True and eval_pan_rels=False as their settings during inference will give you non-duplicated triplets. However, as our deduplication operation is based on masks, the models have to contain mask heads.
2. Using use_mask=True will decelerate your training process as mask heads are also optimized in training and yes, the model will thus focus on mask predictions too.

Since our competition and demos are all meant to get good mask predictions, code for bbox-only models may not be updated instantly. If you want to train and evaluate bbox-only models, you can try to substitute following lines: https://github.com/Jingkang50/OpenPSG/blob/bdd3c548257b99468e7adddfac7f1fd41a4321a7/openpsg/models/relation_heads/psgtr_head.py#1173-1177

        if self.use_mask:
            return det_bboxes, complete_labels, rel_pairs, output_masks, pan_rel_pairs, \
                pan_img, complete_r_labels, complete_r_dists, r_labels, r_dists, pan_masks, rels, pan_labels
        else:
            return det_bboxes, labels, rel_pairs, r_labels, r_dists

with

        if self.use_mask:
            det_bboxes = torch.cat((s_det_bboxes[keep_tri], o_det_bboxes[keep_tri]), 0)
            return det_bboxes, complete_labels, rel_pairs, output_masks, pan_rel_pairs, \
                pan_img, complete_r_labels, complete_r_dists, r_labels, r_dists, pan_masks, rels, pan_labels
        else:
            labels = torch.cat((s_labels, o_labels), 0)
            rel_pairs = torch.arange(len(labels),
                dtype=torch.int).reshape(2, -1).T
            det_bboxes = torch.cat((s_det_bboxes, o_det_bboxes), 0)
            return det_bboxes, labels, rel_pairs, r_labels, r_dists