demo and eval result weird

[ivyone]

I used the pretrained *cityscapes_best.pthl and the setting you provided, but the result of demo and eval is weird.I run it on one single RTX 3080 Ti. Here is the demo setting. 3c3a6ef40cf5:python -u /opt/project/demo.py --demo-image ./images/bad-honnef_000000_000001_leftImg8bit.png --snapshot ./pretrained_models/cityscapes_best.pth --save-dir ./save2/ Using regular batch norm Net built. Net restored. Inference done. Results saved.

Here is the eval setting and result. The IoU is nan.

e6016f3ab652:python -u /opt/project/eval.py --dataset cityscapes --arch network.deepv3.DeepWV3Plus --inference_mode sliding --scales 1.0 --split test --cv_split 0 --dump_images --ckpt_path ./save3 --snapshot ./pretrained_models/cityscapes_best2.pth Using regular batch norm Logging : ./save3/./test/eval_2020_11_28_13_08_49_rank_0.log 11-28 13:08:49.330 Network Arch: network.deepv3.DeepWV3Plus 11-28 13:08:49.331 CV split: 0 11-28 13:08:49.331 Exp_name: . 11-28 13:08:49.331 Ckpt path: ./save3 11-28 13:08:49.331 Scales : 1.0 11-28 13:08:49.331 Inference mode: sliding 11-28 13:08:49.343 Cityscapes-test: 1525 images 11-28 13:08:49.343 Load model file: ./pretrained_models/cityscapes_best2.pth 11-28 13:08:49.355 Trunk: WideResnet38 11-28 13:08:50.277 Global Average Pooling Initialized 11-28 13:23:19.966 Model params = 137.1M 11-28 13:23:20.562 Checkpoint Load Compelete eval test: 0%| | 0/1525 [00:00<?, ?it/s]/opt/project/utils/misc.py:68: RuntimeWarning: invalid value encountered in true_divide (1024, 2048) return np.diag(hist) / (hist.sum(1) + hist.sum(0) - np.diag(hist)) /opt/project/eval.py:464: RuntimeWarning: Mean of empty slice iou = round(np.nanmean(per_class_iu(self.hist)) * 100, 2) Mean IOU: nan: 0%| | 1/1525 [23:00<582:31:58, 1376.06s/it](1024, 2048) Mean IOU: nan: 0%| | 2/1525 [23:02<2:39:23, 6.28s/it] (1024, 2048) Mean IOU: nan: 0%| | 3/1525 [23:12<2:36:13, 6.16s/it](1024, 2048) Mean IOU: nan: 0%| | 4/1525 [23:18<2:33:24, 6.05s/it](1024, 2048) Mean IOU: nan: 0%| | 5/1525 [23:20<2:28:50, 5.88s/it](1024, 2048) Mean IOU: nan: 0%| | 6/1525 [23:30<2:32:13, 6.01s/it](1024, 2048) Mean IOU: nan: 0%| | 7/1525 [23:32<2:25:45, 5.76s/it](1024, 2048) It was still nan until 26%, then I stop the program.

[DesperateMaker]

I have the same problem. I use this code in the docker container.