在自己数据集上训练

[ycdhqzhiai]

有人在自己数据集上训练的吗？效果怎么样，我这边自己标准了detect bbox和车道线，loss从1.0降到0.18(70个epoch)

精度

Lane line Segment: Acc(0.337)    IOU (0.008)  mIOU(0.340)
Detect: P(0.008)  R(0.665)  mAP@0.5(0.260)  mAP@0.5:0.95(0.085)

感觉完全不对

[luckyjohnfeng]

您可以使用已经训练后的权重，更改 end_to_end.pth 到 YOLOP/runs/BddDataset 文件夹，更改为 checkpoint.pth 即可以接着训练，这样进行迁移学习，就可以有一个比较好的开始了。 You could move end_to_end.pth to YOLOP/runs/BddDataset, and change its name to checkpoint.pth, delete the original checkpoint.pth, then do transfer learning. You will have a better start.

Do you know how to custom setting the object detection catagory ?

您知道如何定制设定 目标检测的 分类吗？

[ycdhqzhiai]

谢谢回复 我这边是一个自己标注的7类数据集，训练到200 epoch，检测分支mAP@0.5(0.32)， mAP@0.5:0.95(0.13) 后面做了跟您代码相似操作，去除其他类别只训练car这一类，同样训练了20个epoch mAP@0.5(0.58)， mAP@0.5:0.95(0.32)（log清理掉了） 1.我想问下作者，为什么把所有类别都当做car来训练，不这样做效果如何 2.关于车道线，我这边IOU一直都是在0.01左右，可视化了bdd和我自己标注的label，是一致的，出现这种问题原因有可能是什么

[luckyjohnfeng]

请问您是怎么在YOLOP代码中 设置自己的目标检测类别的呢？

另外车道线分割的的png图片是如何生成的？

代码中，我看到目前只有 可行驶区域的png图片生成代码。

May I know how do you set custom detection category in the YOLOP code ?

And how do you generate the lane segmentation mask png picture ?

I only see the drivable segmentation mask png code in YOLOP.

[ycdhqzhiai]

请问您是怎么在YOLOP代码中 设置自己的目标检测类别的呢？

另外车道线分割的的png图片是如何生成的？

代码中，我看到目前只有 可行驶区域的png图片生成代码。

May I know how do you set custom detection category in the YOLOP code ?

And how do you generate the lane segmentation mask png picture ?

I only see the drivable segmentation mask png code in YOLOP.

import numpy as np
import json
import os
from .AutoDriveDataset import AutoDriveDataset
from .convert import convert, id_dict, my_dict
from tqdm import tqdm
import cv2
single_cls = True       # just detect vehicle

class MCDataset(AutoDriveDataset):
    def __init__(self, cfg, is_train, inputsize, transform=None):
        super().__init__(cfg, is_train, inputsize, transform)
        self.cfg = cfg
        self.is_train = is_train
        self.db = self._get_db()

    def _get_db(self):
        """
        get database from the annotation file

        Inputs:

        Returns:
        gt_db: (list)database   [a,b,c,...]
                a: (dictionary){'image':, 'information':, ......}
        image: image path
        mask: path of the segmetation label
        label: [cls_id, center_x//256, center_y//256, w//256, h//256] 256=IMAGE_SIZE
        """
        print('building database...')
        gt_db = []
        height, width = self.shapes
        if self.is_train:
            txt = 'train.txt'
        else:
            txt = 'val.txt'

        with open(os.path.join(self.data_root, txt), 'r') as f:
            files_list = f.readlines()

        for files in tqdm(list(files_list)):
            files = files.strip()
            image_path = os.path.join(self.img_root, files) + '.jpg'
            #cv_img = cv2.imread(image_path)
            label_path = image_path.replace(".jpg", ".json")
            lane_path =  os.path.join(self.lane_root, files) + '_label.png'
            with open(label_path, 'r') as f:
                label = json.load(f)
            data = label['shapes']
            data = self.filter_data(data)
            gt = np.zeros((len(data), 5))
            for idx, obj in enumerate(data):
                category = obj['label']
                x1 = float(obj['points'][0][0])
                y1 = float(obj['points'][0][1])
                x2 = float(obj['points'][1][0])
                y2 = float(obj['points'][1][1])
                #cv2.retangle(cv_img, (int(x1), int(x2)),(int(y1), int(y2)), 2,(0,0,255),1)
                cls_id = my_dict[category]
                gt[idx][0] = cls_id
                box = convert((width, height), (x1, x2, y1, y2))
                gt[idx][1:] = list(box)
            rec = [{
                'image': image_path,
                'label': gt,
                'lane': lane_path
            }]

            gt_db += rec
        print('database build finish')
        return gt_db

    def filter_data(self, data):
        remain = []
        for obj in data:
            if 'rectangle' == obj['shape_type']:  # obj.has_key('box2d'):
                remain.append(obj)
        return remain

    def evaluate(self, cfg, preds, output_dir, *args, **kwargs):
        """  
        """
        pass

my_dict={'XX0': 0, 'XX1': 1, 'XX2': 2,'XX3': 3,'XX4': 4,'XX5': 5,'XX6': 6}

车道线根据labelme里面提供的脚本。 训练log里面车道线ground truth 训练log里面目标检测ground truth

另外我去除了可行驶区域分支。保留车道线和检测的分支 1.我想问下，为什么把所有类别都当做car来训练，不这样做效果如何

[luckyjohnfeng]

非常感谢您的回复和代码！

应该不是把所有类别都当做car来训练吧，您可以看一下这里 https://github.com/hustvl/YOLOP/issues/21

zhangbaoj commented on 6 Sep 你们的工程是不是把'car', 'bus': , 'truck', 'train'都归为类别0了？

@Riser6

Collaborator Riser6 commented on 6 Sep yes

BDD有10个默认分类：

Object Detection For object detection, 10 classes are evaluated. They are:

0: pedestrian 1: rider 2: car 3: truck 4: bus 5: train 6: motorcycle 7: bicycle 8: traffic light 9: traffic sign

我也想知道 如何在YOLOP中 设置自己的分类。

再次感谢！！！

[luckyjohnfeng]

关于如何生成 车道线的mask png 图片，您可以提供再详细一些的指引吗？

我调整了可行驶区域的mask png 图片分割代码，目前可以生成很细的车道线

我不知道如何生成训练集中的比较粗的车道线

[luckyjohnfeng]

您可以参考一下这个链接 里面有提到类别： https://github.com/hustvl/YOLOP/issues/56

[ycdhqzhiai]

import os
import cv2
import tqdm
import numpy as np
import pdb
import json, argparse

def draw(im,line,idx,show=True):
    '''
    Generate the segmentation label according to json annotation
    '''
    for i in range(len(line)-1):
        pt0 = (int(line[i][0]), int(line[i][1])) 
        pt1 = (int(line[i + 1][0]), int(line[i + 1][1]))
        cv2.line(im,pt0,pt1,(idx,),thickness = 16)

def generate_segmentation(root, lines, names, mode):
    train_gt_fp = open(os.path.join(root, mode + '_gt.txt'),'w')
    for i in tqdm.tqdm(range(len(names))):
        line_info = lines[i]
        label_path = os.path.join(root, 'seg_lbel', mode)  + '/' + names[i]+'.png'
        image_path = os.path.join(root, 'images', mode)  + '/' + names[i]+'.jpg'
        if not os.path.exists(os.path.split(label_path)[0]):
            os.makedirs(os.path.split(label_path)[0])        
        label = np.zeros((1080,1920),dtype=np.uint8)
        bin_label = [0,0,0,0]
        for j in range(len(line_info)):
            index = int(float(line_info[j]['label'][0]))
            points = line_info[j]['points']
            bin_label[index] = 1
            draw(label,points,index + 1)
        # print(bin_label)
        # cv2.imshow('label', label)
        # cv2.waitKey(0)
        cv2.imwrite(label_path,label)
        # print(image_path, label_path)
        train_gt_fp.write(image_path + ' ' + label_path + ' '+' '.join(list(map(str,bin_label))) + '\n')
    train_gt_fp.close()

def get_mc_list(root, mode):
    '''
    Get all the files' names from the json annotation
    '''
    label_json_all = []
    with open(os.path.join(root, mode + '.txt'), 'r') as f:
        label_list = f.readlines()      
    for l in label_list:
        lab_path = os.path.join(root,'labels', mode) + '/' + l.strip() + '.json'
        with open(lab_path,'r') as load_f:
            load_dict = json.load(load_f) 
        label_json = [load_dict]
        label_json_all += label_json

    names = [l.strip() for l in label_list]
    lanes = [l['shapes'] for l in label_json_all]

    return names,lanes

def get_args():
    parser = argparse.ArgumentParser()
    parser.add_argument('--root', required=True, help='The root of the Tusimple dataset')
    return parser

if __name__ == "__main__":
    args = get_args().parse_args()

    for mode in ['train', 'val']:
        names,lanes = get_mc_list(args.root, mode)
        print(len(names))
        generate_segmentation(args.root, lanes, names, mode)

把idx换成255就可以生成了

[luckyjohnfeng]

非常非常感谢！！！！ 我明早尝试一下！

[Nanjiangang]

您可以参考一下这个链接 里面有提到类别： #56

请问您知道怎么才能目标检测多个类别吗

[luckyjohnfeng]

嗯，我大概知道，就是按照这个 #56的链接 来修改 modify model.nc in train.py to be 41, change the single_cls to be False in bdd.py 再修改convert.py 你看一下 bdd.py的代码 会更清楚。

我想问一下， 如何使用yolop已经训练好的权重，来进行迁移学习 来检测不同的分类？

[Nanjiangang]

我已经按那个修改过了， 麻烦你看一下是这样设置吗？ model.nc=41 in train.py, single_cls = False in bdd.py，id_dict = {'xx':0, 'xx': 1,…… 'xx': 40} in convert.py 但是会报错 Traceback (most recent call last): File "tools/train.py", line 395, in main() File "tools/train.py", line 323, in main epoch, num_batch, num_warmup, writer_dict, logger, device, rank) File "F:\yoloppp\YOLOP-main\lib\core\function.py", line 77, in train total_loss, head_losses = criterion(outputs, target, shapes,model) File "D:\anaconda\envs\yolop\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "F:\yoloppp\YOLOP-main\lib\core\loss.py", line 50, in forward total_loss, head_losses = self._forward_impl(head_fields, head_targets, shapes, model) File "F:\yoloppp\YOLOP-main\lib\core\loss.py", line 107, in _forward_impl t[range(n), tcls[i]] = cp IndexError: index 26 is out of bounds for dimension 1 with size 1

关于您说的使用yolop已经训练好的权重，来进行迁移学习来检测不同的分类是指应用在不同数据集上还是增加检测头？

[luckyjohnfeng]

我这边目前在标注数据，大约周末回开始训练，所以目前我并不知道改动后报错的原因，建议您在pycharm里设置断点， 调试程序。我这边是自己公司采集的数据，进行标注的。目标检测类别有调整，车道线和可行驶区域不变。我想使用YOLOP在BDD100K上的训练权重，进行迁移学习。

[luckyjohnfeng]

您要是找到改动分类后报错的原因，也请告诉我一下，谢谢！

[luckyjohnfeng]

convert.py 里面 有一个 id_dict_single = {'car': 0, 'bus': 1, 'truck': 2,'train': 3} 实际上YOLOP是过滤了id_dict = {'xx':0, 'xx': 1,…… 'xx': 40}的分类，把所有的car, bus, truck, train 通过id_dict_single 归为1个car的类别，进行目标检测，我想 model.nc=1 in train.py 就是对应这一个car的类别， 这部分代码 在bdd.py中有体现。您可以看一下是不是id_dict_single 这里 没有设置好。

[Nanjiangang]

当single_cls = False in bdd.py时，就不需要id_dict_single={'car': 0, 'bus': 1, 'truck': 2,'train': 3}来过滤了，目前来看我的错误好像是loss.py中 t = torch.full_like(ps[:, 5:], cn, device=device) 这个矩阵只有一列，不对应41分类；我需要后续再看一下是哪里的问题，您有什么进展也请告诉我一下，谢谢

[hetao828]

我下午遇到了相同的错误，修改了网络结构。然后报错清除，正常训练。但是我的精确率训练一直很低。不知道二位能否建个群讨论一下！谢谢

[Nanjiangang]

非常感谢，修改网络结构后可以work，我还没有进行大规模训练，所以不清楚会不会随着epoch的增加精确率也会提高？可以群828638733大家一起讨论下

[luckyjohnfeng]

我已经申请加入qq群了，上午我在尝试把CVAT的XML标注，转换成YOLOP需要的mask，然后让同事开始标注，计划周末或下周一开始训练模型。很想一次训练成功！希望与你们交流一下经验。：）

[FancyFai]

有人在自己数据集上训练的吗？效果怎么样，我这边自己标准了detect bbox和车道线，loss从1.0降到0.18(70个epoch)

精度

Lane line Segment: Acc(0.337)    IOU (0.008)  mIOU(0.340)
Detect: P(0.008)  R(0.665)  mAP@0.5(0.260)  mAP@0.5:0.95(0.085)

感觉完全不对

楼主，你好我也是出现这样的情况，我只是把类别放到检测14个类，训练100epoch还是准确率低的奇怪，而且可视化出来的时候，detect的图像出现很多重叠框，感觉NMS没起到作用，请问楼主后来解决这个问题了吗

[ycdhqzhiai]

没弄了，我感觉多分类有问题，单分类map也不高，不过能正常出框，但是多分类就完全不行我，后面就没弄了

[FancyFai]

没弄了，我感觉多分类有问题，单分类map也不高，不过能正常出框，但是多分类就完全不行我，后面就没弄了

好的，谢谢回复啦，我也是多分类map不高

[tuvshu99]

May i ask some questions regarding to multi class object detection head? I am training the model with multi class object detection. In early epochs, detection result is something like every object have detected as a few classes. How many epochs are good for multi class detection result? How did you configure the thresholds for detection? Also, the test result after 10 epochs, detection look like this image. Is this normal? Ground truth Detected objects

[dinarkino]

Hey @tuvshu99, interesting results! Have you succeeded with multi-class detection? Your prediction looks reasonable for the 10th epoch. In my case the prediction quality is very low. I'm trying to figure out where the problem. Have you changed something in the code? Did you train just for detection or for detection + segmentation task? Also did you start from pre-trained weights (at least part of them until detect layer) or did you train the model from scratch?

[tuvshu99]

I have trained the multiclass model from the scratch. For the detection result, i think i changed some thresholds but i don't remember now. For the training and inference, i have changed the model and config.

[dinarkino]

Thank you for the answer! If you saved at least some metrics at the end of training or configs/changed model code, it would be very helpful for me.

[feichaiyu]

关于如何生成 车道线的mask png 图片，您可以提供再详细一些的指引吗？

我调整了可行驶区域的mask png 图片分割代码，目前可以生成很细的车道线

我不知道如何生成训练集中的比较粗的车道线

hi你好，请问你解决了这个问题么？ 无法生成带有类别的车道线mask

[Ri-Bai]

谢谢回复 我这边是一个自己标注的7类数据集，训练到200 epoch，检测分支mAP@0.5(0.32)， mAP@0.5:0.95(0.13) 后面做了跟您代码相似操作，去除其他类别只训练car这一类，同样训练了20个epoch mAP@0.5(0.58)， mAP@0.5:0.95(0.32)（log清理掉了） 1.我想问下作者，为什么把所有类别都当做car来训练，不这样做效果如何 2.关于车道线，我这边IOU一直都是在0.01左右，可视化了bdd和我自己标注的label，是一致的，出现这种问题原因有可能是什么

我遇到了同样的问题，训练自己数据集（多类别）精度较低，请问您使用预训练权重了吗，如何解决的，我使用预训练文件一直报错

[Ri-Bai]

end_to_end.pth to YOLOP/runs/BddDataset, and change its nam

多类别能用预训练权重吗，我的报错了RuntimeError: Error(s) in loading state_dict for MCnet: size mismatch for model.24.m.0.weight: copying a param with shape torch.Size([18, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([21, 128, 1, 1]). size mismatch for model.24.m.0.bias: copying a param with shape torch.Size([18]) from checkpoint, the shape in current model is torch.Size([21]). size mismatch for model.24.m.1.weight: copying a param with shape torch.Size([18, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([21, 256, 1, 1]). size mismatch for model.24.m.1.bias: copying a param with shape torch.Size([18]) from checkpoint, the shape in current model is torch.Size([21]). size mismatch for model.24.m.2.weight: copying a param with shape torch.Size([18, 512, 1, 1]) from checkpoint, the shape in current model is torch.Size([21, 512, 1, 1]). size mismatch for model.24.m.2.bias: copying a param with shape torch.Size([18]) from checkpoint, the shape in current model is torch.Size([21]). 请问有没有建议或解决方法

[tfidmmatthew]

@luckyjohnfeng Does it work?

import os
import cv2
import tqdm
import numpy as np
import pdb
import json, argparse

def draw(im,line,idx,show=True):
    '''
    Generate the segmentation label according to json annotation
    '''
    for i in range(len(line)-1):
        pt0 = (int(line[i][0]), int(line[i][1])) 
        pt1 = (int(line[i + 1][0]), int(line[i + 1][1]))
        cv2.line(im,pt0,pt1,(idx,),thickness = 16)

def generate_segmentation(root, lines, names, mode):
    train_gt_fp = open(os.path.join(root, mode + '_gt.txt'),'w')
    for i in tqdm.tqdm(range(len(names))):
        line_info = lines[i]
        label_path = os.path.join(root, 'seg_lbel', mode)  + '/' + names[i]+'.png'
        image_path = os.path.join(root, 'images', mode)  + '/' + names[i]+'.jpg'
        if not os.path.exists(os.path.split(label_path)[0]):
            os.makedirs(os.path.split(label_path)[0])        
        label = np.zeros((1080,1920),dtype=np.uint8)
        bin_label = [0,0,0,0]
        for j in range(len(line_info)):
            index = int(float(line_info[j]['label'][0]))
            points = line_info[j]['points']
            bin_label[index] = 1
            draw(label,points,index + 1)
        # print(bin_label)
        # cv2.imshow('label', label)
        # cv2.waitKey(0)
        cv2.imwrite(label_path,label)
        # print(image_path, label_path)
        train_gt_fp.write(image_path + ' ' + label_path + ' '+' '.join(list(map(str,bin_label))) + '\n')
    train_gt_fp.close()

def get_mc_list(root, mode):
    '''
    Get all the files' names from the json annotation
    '''
    label_json_all = []
    with open(os.path.join(root, mode + '.txt'), 'r') as f:
        label_list = f.readlines()      
    for l in label_list:
        lab_path = os.path.join(root,'labels', mode) + '/' + l.strip() + '.json'
        with open(lab_path,'r') as load_f:
            load_dict = json.load(load_f) 
        label_json = [load_dict]
        label_json_all += label_json

    names = [l.strip() for l in label_list]
    lanes = [l['shapes'] for l in label_json_all]

    return names,lanes

def get_args():
    parser = argparse.ArgumentParser()
    parser.add_argument('--root', required=True, help='The root of the Tusimple dataset')
    return parser

if __name__ == "__main__":
    args = get_args().parse_args()

    for mode in ['train', 'val']:
        names,lanes = get_mc_list(args.root, mode)
        print(len(names))
        generate_segmentation(args.root, lanes, names, mode)

把idx换成255就可以生成了

@luckyjohnfeng Does it work?

[LuthraBhomik]

Has anyone been able to train a model on custom dataset (with number of classes e.g. 3) using YOLOP pretrained model (number of classes = 1). I am able to train a model on custom dataset from scratch but not able to train a model using pretrained model. Kindly update, if anyone is able to do it.