please help me! I meet a problem when training my own dataset

[electronicYH]

I have a dataset with 14 classes(include 13 kinds object classes and 1 kind background) In the dataset, background'id is set to 0, and the others classes' id is 1-13, in the label picture, the background pix value is 0, the others classes pix value is 1-13 The problem is that, when training the model, the IoU of class 1 is always 0.0, Overall Acc: 0.7006523041507537 INFO:ptsemseg:Overall Acc: : 0.7006523041507537 Mean Acc : 0.2108480790158249 INFO:ptsemseg:Mean Acc : : 0.2108480790158249 FreqW Acc : 0.5198288277808017 INFO:ptsemseg:FreqW Acc : : 0.5198288277808017 Mean IoU : 0.14262183776063359 INFO:ptsemseg:Mean IoU : : 0.14262183776063359 INFO:ptsemseg:0: 0.6279548955918839 INFO:ptsemseg:1: 0.0 INFO:ptsemseg:2: 0.0 INFO:ptsemseg:3: 0.6441913898204116 INFO:ptsemseg:4: 0.5818516148968814 INFO:ptsemseg:5: 1.0196236772931845e-06 INFO:ptsemseg:6: 0.0 INFO:ptsemseg:7: 0.0 INFO:ptsemseg:8: 0.0 INFO:ptsemseg:9: 0.0 INFO:ptsemseg:10: 8.497095538252379e-05 INFO:ptsemseg:11: 0.0 INFO:ptsemseg:12: 0.0

futhermore, sometimes it get error: INFO:ptsemseg:Iter 1500 Val Loss: 1.6997 _/home/new02/yh/FCHarDNet/ptsemseg/metrics.py:34: RuntimeWarning: invalid value encountered in truedivide iu = np.diag(hist) / (hist.sum(axis=1) + hist.sum(axis=0) - np.diag(hist)) Overall Acc: 0.6941342720781378 INFO:ptsemseg:Overall Acc: : 0.6941342720781378 Mean Acc : 0.20532103027188917 INFO:ptsemseg:Mean Acc : : 0.20532103027188917 FreqW Acc : 0.5145511719663355 INFO:ptsemseg:FreqW Acc : : 0.5145511719663355 Mean IoU : 0.15183979860183058 INFO:ptsemseg:Mean IoU : : 0.15183979860183058 INFO:ptsemseg:0: 0.6409373749170948 INFO:ptsemseg:1: nan WARNING:root:NaN or Inf found in input tensor. INFO:ptsemseg:2: 0.0 INFO:ptsemseg:3: 0.6054743860395182 INFO:ptsemseg:4: 0.5756210137925749 INFO:ptsemseg:5: 1.0195457108221718e-06 INFO:ptsemseg:6: 0.0 INFO:ptsemseg:7: 0.0 INFO:ptsemseg:8: 0.0 INFO:ptsemseg:9: 0.0 INFO:ptsemseg:10: 4.378892706825406e-05 INFO:ptsemseg:11: 0.0 INFO:ptsemseg:12: 0.0 …… INFO:ptsemseg:Iter 1800 Val Loss: 1.3872 Overall Acc: 0.7570247568021282 INFO:ptsemseg:Overall Acc: : 0.7570247568021282 Mean Acc : 0.22404660812178812 INFO:ptsemseg:Mean Acc : : 0.22404660812178812 FreqW Acc : 0.59153389710979 INFO:ptsemseg:FreqW Acc : : 0.59153389710979 Mean IoU : 0.17551320192855324 INFO:ptsemseg:Mean IoU : : 0.17551320192855324 INFO:ptsemseg:0: 0.733411667979755 INFO:ptsemseg:1: nan WARNING:root:NaN or Inf found in input tensor. INFO:ptsemseg:2: 0.0 INFO:ptsemseg:3: 0.7271575023121397 INFO:ptsemseg:4: 0.6455313922516163 INFO:ptsemseg:5: 0.0 INFO:ptsemseg:6: 0.0 INFO:ptsemseg:7: 0.0 INFO:ptsemseg:8: 0.0 INFO:ptsemseg:9: 0.0 INFO:ptsemseg:10: 5.406602233699094e-05 INFO:ptsemseg:11: 3.7945767908505163e-06 INFO:ptsemseg:12: 0.0

I modified the cityscapes_loader.py: colors = [ #[ 0, 0, 0], [128, 64, 128], [244, 35, 232], [70, 70, 70], [102, 102, 156], [190, 153, 153], [153, 153, 153], [250, 170, 30], [220, 220, 0], [107, 142, 35], [152, 251, 152], [0, 130, 180], [220, 20, 60], [255, 0, 0],

[0, 0, 142],

    #[0, 0, 70],
    #[0, 60, 100],
    #[0, 80, 100],
    #[0, 0, 230],
    #[119, 11, 32],
]

label_colours = dict(zip(range(13), colors))

   self.void_classes = [0]
    self.valid_classes = [
        1,
        2,
        3,
        4,
        5,
        6,
        7,
        8,
        9,
        10,
        11,
        12,
        13,
    ]

    self.class_names = [
        "background",
        "class1",
        "class2",
        "class3",
        "class4",
        "class5",
        "class6",
        "class7",
        "class8",
        "class9",
        "class10",
        "class12",
        "class13",
    ]

    self.ignore_index = 250
    self.class_map = dict(zip(self.valid_classes, range(13)))

please help me! How can I fix the problem?

[EchoAmor]

@electronicYH Hello,I have met the same problem with you ,have u solved the problem?

[EchoAmor]

@PingoLH Hello , I have modified your sunrgbd_dataload.py code to work with ScanNet,and when I run it on ScanNet dataset, Once iterates five hundred times or the number of iterations is a multiple of five hundred, an warning is reported ,

And when it pass 500,I got the validation results like this:

Do u know where the problem is ? Can u help me please? This work is very important to me .Thanks a lot!