How to train custom dataset ?

[devendraswamy]

Great work on 3d object detection , could you please help me to create and train a custom dataset , Thanking you in advance.

[ruhyadi]

Thank you for your interest. Sorry for late response. In simple word you need convert your own dataset label to YOLO3D label, like this (see this) :

        label = {
            'Class': Class,
            'Box_2D': Box_2D,
            'Dimensions': Dimension,
            'Alpha': Alpha,
            'Orientation': Orientation,
            'Confidence': Confidence
        }

In the KITTI dataset you will get the following labels for each image (in .txt file):

labels = {
    'type': 'car',                   # Describes the type of object: 'Car','Pedestrian', etc
    'truncated': 0,                  # 0 to 1, Truncated refers to the object leaving image boundaries
    'occluded': 0,                   # indicating occlusion state: 0 = fully visible, 1 = partly occluded 2 largely occluded, 3 = unknown
    'alpha': 1,                      # Observation angle of object, ranging [-pi..pi]
    'bbox': [50, 25, 25, 50],        # 2D bounding box of object in the image (0-based index): contains left, top, right, bottom pixel coordinates
    'dimensions': [1.2, 1.5, 1.2],   # 3D object dimensions: height, width, length (in meters)
    'location': [2.5, 4.5, 3.3],     # 3D object location x,y,z in camera coordinates (in meters)
    'rotation_y':  0.75,             # Rotation ry around Y-axis in camera coordinates [-pi..pi]
    'score': 0,                      # Only for results: Float
}

and you have to convert those labels to YOLO3D labels.

For now I'm working on the Lyft dataset (the label format is the same as nuScene). I'll be back in a few days to add new dataset capabilites.

[devendraswamy]

Thank you so much for your valuable response. Could you please let me know which label tool I have to use for labelling the images ?Thanking you in advance

[ruhyadi]

You can't label all images as a 3D object detection dataset, it's not like a 2D detection dataset (COCO, etc.). In the 3d object detection dataset you must have a calibration matrix (see this) for each image and a 3D bounding box for each object in the image. Making a 3D bounding box itself must have information about the rotation (yaw of the object), dimensions, location of object in the 3D world, etc. I recommend that you use existing 3D object detection datasets, such as KITTI, nuScenes, and Lyft.

[devendraswamy]

Thank you for your support but here the problem is custom object , that shouldn't be there in open source datasets like kitti, lyft . actually my class is different that's why i request to you , is their any possibility to make a dataset in the format of kitti for this project , kindly help me.

[ruhyadi]

I'm really sorry, didn't reply to this issue for a few months.

For custom 3D object detection datasets, you can try CVAT (Computer Vision Annotation Tools) from OpenVino. CVAT also supports the semi-automatic labeling method, so you can use this model (yolo3d) or better ones like SMOKE, RTM3D, etc. to generate a 3D bounding-box on your custom dataset. If the results are not very accurate on a custom dataset, you can also manually label it. Please read cvat documentations.

I will close this issue now. If you have further questions, don't hesitate to open again or create a new issue.

[devendraswamy]

Thank you for your response and valuable feedback. I'll try it once.

On Sun, May 29, 2022, 19:16 Didi Ruhyadi @.***> wrote:

Closed #6 https://github.com/ruhyadi/YOLO3D/issues/6 as completed.

— Reply to this email directly, view it on GitHub https://github.com/ruhyadi/YOLO3D/issues/6#event-6698320524, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALLD6LVKPMSXIMXMRVEPBV3VMNYJTANCNFSM5RRSEH2Q . You are receiving this because you commented.Message ID: @.***>