Panoptic demo doesn't provide panoptic predictions

[saltoricristiano]

Hi!

Thanks for releasing the code of this very nice work!! I'm currently trying to run your demo on a cityscapes image. Everything works except that setting task to panoptic doesn't provide also the semantic predictions for the "stuff" class but only instances. Here the results:

And here the results with task=semantic

I obtained it by running the demo code with checkpoint https://shi-labs.com/projects/oneformer/cityscapes/250_16_swin_l_oneformer_cityscapes_90k.pth

Is there something wrong from my side? During the demo I get

/home/csaltori/miniconda3/envs/detectron/lib/python3.8/site-packages/detectron2/structures/image_list.py:88: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  max_size = (max_size + (stride - 1)) // stride * stride
/home/csaltori/Projects/OneFormer/demo/../oneformer/modeling/transformer_decoder/position_encoding.py:44: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  dim_t = self.temperature ** (2 * (dim_t // 2) / self.num_pos_feats)
/home/csaltori/Projects/OneFormer/demo/../oneformer/oneformer_model.py:448: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  topk_indices = topk_indices // self.sem_seg_head.num_classes
[06/09 10:02:00 detectron2]: ../images/image-2.png: detected 9 instances in 6.80s
100%|████████████████████████████████████████████████████████████████████████████████| 1/1 [00:09<00:00,  9.88s/it]

Thanks in advance!

[praeclarumjj3]

Hi @saltoricristiano, please take a look at #60. Your issue seems similar to that one.