Data set processing?

[xiaoyufenfei]

Hello @mapleneverfade What you mean is that all data sets are placed as follows:

• datasets

  • VOC2012

  • JPEGImages

  • gtFine

  • train

    • image.txt
    • label.txt

  • val

    • image.txt
    • label.txt

  • test

    • image.txt
    • label.txt

  • Cityscapes

  • JPEGImages

  • gtFine

  • train

    • image.txt
    • label.txt

  • val

    • image.txt
    • label.txt

  • test

    • image.txt
    • label.txt But I don't understand another question, you gtFine in cityscapes using _labelTrainIds.png or _labelIds.png? I can't figure it out. Can you help me? Thank you very much!

[mapleneverfade]

I'm not sure that i know what puzzles you. But gtFine should be the gray scale image.

[xiaoyufenfei]

@mapleneverfade Thank you! I am sure that the cityscapes-gtFine folder is a grayscale image, Just, I'm not sure if you are using _labelIds.png or the processed _labelTrainIds.png?

When I used *_labelTrainIds.png, I encountered the following problem:

CUDA_VISIBLE_DEVICES=0 python3 train.py --datadir ./data/cityscapes/ --savedir ./save_models/segnet/ --model segnet --num-classes 20 --num-epochs 150 --batch-size 2 --steps-loss 50 --num-workers 4 --epoch-save 10 ------------ Options ------------- batch_size: 2 cuda: True datadir: ./data/cityscapes/ epoch_save: 10 iouTrain: True iouVal: True lr: 0.0005 model: segnet num_classes: 20 num_epochs: 150 num_workers: 4 pretrained: ./pre_trained/~~~.pth resume: False savedir: ./save_models/segnet/ state: None steps_loss: 50 -------------- End ---------------- RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/aten/src/THC/generic/THCStorage.cpp:36 /pytorch/aten/src/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [5,0,0], thread: [862,0,0] Assertion t >= 0 && t < n_classes failed. /pytorch/aten/src/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [5,0,0], thread: [863,0,0] Assertion t >= 0 && t < n_classes failed.

I am not sure if it is a dataset label problem. When I try to replace the label in cityscapes-gtFine with *_labelIds.png, i encounter the same problem?

I can't figure it out. Can you help me? Thank you very much!

[mapleneverfade]

Have you ever notice this?

[xiaoyufenfei]

@mapleneverfade My dataset is already processed. I use the Cityscapesscripts script to generate _labelTrainIds.png based on _labelIds.png. I am not sure if you are using _labelIds.png or _labelTrainIds.png in gtFine while you are training. The directory ./utils/cityscapes/helps/labels is part of Cityscapesscripts, I have dealt with it, and I set num_classes=20 in Training, but when I store the label(_labelIds.png or _labelTrainIds.png) in gtFine directory , the program still has the above error. I can't figure it out. Can you help me? Thank you very much!

[mapleneverfade]

Actually i use https://github.com/wkentaro/labelme to generate labels. Please check out whether the label is range from 0 to 19.

[xiaoyufenfei]

@mapleneverfade I understand what you mean, you mean that the data of ./data/dataset/gtFine is processed by https://github.com/wkentaro/labelme, but the official dataset of PasCalVOC2012 has been processed, http://host.robots.ox.ac.uk/pascal/VOC/voc2012/index.html， Can this not be used directly?

Another Cityscapes is also an officially processed dataset, but always need us use the script https://github.com/kinglintianxia/cityscapesScripts to process _labelIds.png to _labelTrainIds.png, I want to know how you train the network on cityscapes ? It's impossible to re-label cityscapes. This is not in line with the principle of using public benchmark data sets, unless it is your own new data set? I can't figure it out. Can you help me? Thank you very much!