Sylvus
commented
7 years ago I have/had a similar issue. First of all, setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 helped me with the memory problem. I was able to validate with a size of 2048 on a K80 whereas before I would always receive an error.
The reported accuracy and mean iou are still very low but the output looks much better now. So I think there might be a mistake in the way the accuracy/iou values are calculated.
Hope that helps!
shipengai
I use 0.93 mxnet and use scripts below:
python issegm/voc.py --gpus 0,1 --split train --data-root /cityscapes --output output --model cityscapes_rna-a1_cls19_s8 --batch-images 10 --crop-size 500 --origin-size 2048 --scale-rate-range 0.7,1.3 --weights models/ilsvrc-cls_rna-a1_cls1000_ep-0001.params --lr-type fixed --base-lr 0.0016 --to-epoch 140 --kvstore local --prefetch-threads 8 --prefetcher process --cache-images 0 --backward-do-mirror
python issegm/voc.py --gpus 0,1 --split train --data-root /cityscapes --output output --model cityscapes_rna-a1_cls19_s8_x1-140 --batch-images 10 --crop-size 500 --origin-size 2048 --scale-rate-range 0.7,1.3 --weights output/cityscapes_rna-a1_cls19_s8_ep-0140.params --lr-type linear --base-lr 0.0008 --to-epoch 64 --kvstore local --prefetch-threads 8 --prefetcher process --cache-images 0 --backward-do-mirror
python issegm/voc.py --data-root /cityscapes --output output --phase val --weights output/cityscapes_rna-a1_cls19_s8_x1-140_ep-0064.params --split val --test-scales 1024 --test-flipping --gpus 1
Host pixel acc: 37.65%, mean acc: 5.26%, mean iou: 1.98%
I know there is different from orignal command that use test-scales 2048, but i can load that with 12 GB memory, so i reduce this parameter.
I download the model and check md5, everything is ok.