Custom Dataset not implemented [Bug]

[kbratsy]

Prerequisite

• [X] I have searched Issues and Discussions but cannot get the expected help.
• [X] I have read the FAQ documentation but cannot get the expected help.
• [X] The bug has not been fixed in the latest version (dev-1.x) or latest version (dev-1.0).

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

main branch https://github.com/open-mmlab/mmdetection3d

Environment

Name Version Build Channel

mmcv 2.0.0 pypi_0 pypi mmdet 3.0.0 pypi_0 pypi mmdet3d 1.1.1 dev_0 mmengine 0.7.4 pypi_0 pypi numpy 1.24.3 pypi_0 pypi open3d 0.17.0 pypi_0 pypi openmim 0.3.7 pypi_0 pypi python 3.8.16 h4de0772_1_cpython conda-forge

Reproduces the problem - code sample

# Copyright (c) OpenMMLab. All rights reserved.
import argparse
from os import path as osp

from tools.dataset_converters import indoor_converter as indoor
from tools.dataset_converters import kitti_converter as kitti
from tools.dataset_converters import lyft_converter as lyft_converter
from tools.dataset_converters import nuscenes_converter as nuscenes_converter
from tools.dataset_converters import semantickitti_converter
from tools.dataset_converters.create_gt_database import (
    GTDatabaseCreater, create_groundtruth_database)
from tools.dataset_converters.update_infos_to_v2 import update_pkl_infos

def kitti_data_prep(root_path,
                    info_prefix,
                    version,
                    out_dir,
                    with_plane=False):
    """Prepare data related to Kitti dataset.

    Related data consists of '.pkl' files recording basic infos,
    2D annotations and groundtruth database.

    Args:
        root_path (str): Path of dataset root.
        info_prefix (str): The prefix of info filenames.
        version (str): Dataset version.
        out_dir (str): Output directory of the groundtruth database info.
        with_plane (bool, optional): Whether to use plane information.
            Default: False.
    """
    kitti.create_kitti_info_file(root_path, info_prefix, with_plane)
    kitti.create_reduced_point_cloud(root_path, info_prefix)

    info_train_path = osp.join(out_dir, f'{info_prefix}_infos_train.pkl')
    info_val_path = osp.join(out_dir, f'{info_prefix}_infos_val.pkl')
    info_trainval_path = osp.join(out_dir, f'{info_prefix}_infos_trainval.pkl')
    info_test_path = osp.join(out_dir, f'{info_prefix}_infos_test.pkl')
    update_pkl_infos('kitti', out_dir=out_dir, pkl_path=info_train_path)
    update_pkl_infos('kitti', out_dir=out_dir, pkl_path=info_val_path)
    update_pkl_infos('kitti', out_dir=out_dir, pkl_path=info_trainval_path)
    update_pkl_infos('kitti', out_dir=out_dir, pkl_path=info_test_path)
    create_groundtruth_database(
        'KittiDataset',
        root_path,
        info_prefix,
        f'{info_prefix}_infos_train.pkl',
        relative_path=False,
        mask_anno_path='instances_train.json',
        with_mask=(version == 'mask'))

def nuscenes_data_prep(root_path,
                       info_prefix,
                       version,
                       dataset_name,
                       out_dir,
                       max_sweeps=10):
    """Prepare data related to nuScenes dataset.

    Related data consists of '.pkl' files recording basic infos,
    2D annotations and groundtruth database.

    Args:
        root_path (str): Path of dataset root.
        info_prefix (str): The prefix of info filenames.
        version (str): Dataset version.
        dataset_name (str): The dataset class name.
        out_dir (str): Output directory of the groundtruth database info.
        max_sweeps (int, optional): Number of input consecutive frames.
            Default: 10
    """
    nuscenes_converter.create_nuscenes_infos(
        root_path, info_prefix, version=version, max_sweeps=max_sweeps)

    if version == 'v1.0-test':
        info_test_path = osp.join(out_dir, f'{info_prefix}_infos_test.pkl')
        update_pkl_infos('nuscenes', out_dir=out_dir, pkl_path=info_test_path)
        return

    info_train_path = osp.join(out_dir, f'{info_prefix}_infos_train.pkl')
    info_val_path = osp.join(out_dir, f'{info_prefix}_infos_val.pkl')
    update_pkl_infos('nuscenes', out_dir=out_dir, pkl_path=info_train_path)
    update_pkl_infos('nuscenes', out_dir=out_dir, pkl_path=info_val_path)
    create_groundtruth_database(dataset_name, root_path, info_prefix,
                                f'{info_prefix}_infos_train.pkl')

def lyft_data_prep(root_path, info_prefix, version, max_sweeps=10):
    """Prepare data related to Lyft dataset.

    Related data consists of '.pkl' files recording basic infos.
    Although the ground truth database and 2D annotations are not used in
    Lyft, it can also be generated like nuScenes.

    Args:
        root_path (str): Path of dataset root.
        info_prefix (str): The prefix of info filenames.
        version (str): Dataset version.
        max_sweeps (int, optional): Number of input consecutive frames.
            Defaults to 10.
    """
    lyft_converter.create_lyft_infos(
        root_path, info_prefix, version=version, max_sweeps=max_sweeps)
    if version == 'v1.01-test':
        info_test_path = osp.join(root_path, f'{info_prefix}_infos_test.pkl')
        update_pkl_infos('lyft', out_dir=root_path, pkl_path=info_test_path)
    elif version == 'v1.01-train':
        info_train_path = osp.join(root_path, f'{info_prefix}_infos_train.pkl')
        info_val_path = osp.join(root_path, f'{info_prefix}_infos_val.pkl')
        update_pkl_infos('lyft', out_dir=root_path, pkl_path=info_train_path)
        update_pkl_infos('lyft', out_dir=root_path, pkl_path=info_val_path)

def scannet_data_prep(root_path, info_prefix, out_dir, workers):
    """Prepare the info file for scannet dataset.

    Args:
        root_path (str): Path of dataset root.
        info_prefix (str): The prefix of info filenames.
        out_dir (str): Output directory of the generated info file.
        workers (int): Number of threads to be used.
    """
    indoor.create_indoor_info_file(
        root_path, info_prefix, out_dir, workers=workers)
    info_train_path = osp.join(out_dir, f'{info_prefix}_infos_train.pkl')
    info_val_path = osp.join(out_dir, f'{info_prefix}_infos_val.pkl')
    info_test_path = osp.join(out_dir, f'{info_prefix}_infos_test.pkl')
    update_pkl_infos('scannet', out_dir=out_dir, pkl_path=info_train_path)
    update_pkl_infos('scannet', out_dir=out_dir, pkl_path=info_val_path)
    update_pkl_infos('scannet', out_dir=out_dir, pkl_path=info_test_path)

def s3dis_data_prep(root_path, info_prefix, out_dir, workers):
    """Prepare the info file for s3dis dataset.

    Args:
        root_path (str): Path of dataset root.
        info_prefix (str): The prefix of info filenames.
        out_dir (str): Output directory of the generated info file.
        workers (int): Number of threads to be used.
    """
    indoor.create_indoor_info_file(
        root_path, info_prefix, out_dir, workers=workers)
    splits = [f'Area_{i}' for i in [1, 2, 3, 4, 5, 6]]
    for split in splits:
        filename = osp.join(out_dir, f'{info_prefix}_infos_{split}.pkl')
        update_pkl_infos('s3dis', out_dir=out_dir, pkl_path=filename)

def sunrgbd_data_prep(root_path, info_prefix, out_dir, workers):
    """Prepare the info file for sunrgbd dataset.

    Args:
        root_path (str): Path of dataset root.
        info_prefix (str): The prefix of info filenames.
        out_dir (str): Output directory of the generated info file.
        workers (int): Number of threads to be used.
    """
    indoor.create_indoor_info_file(
        root_path, info_prefix, out_dir, workers=workers)
    info_train_path = osp.join(out_dir, f'{info_prefix}_infos_train.pkl')
    info_val_path = osp.join(out_dir, f'{info_prefix}_infos_val.pkl')
    update_pkl_infos('sunrgbd', out_dir=out_dir, pkl_path=info_train_path)
    update_pkl_infos('sunrgbd', out_dir=out_dir, pkl_path=info_val_path)

def waymo_data_prep(root_path,
                    info_prefix,
                    version,
                    out_dir,
                    workers,
                    max_sweeps=5):
    """Prepare the info file for waymo dataset.

    Args:
        root_path (str): Path of dataset root.
        info_prefix (str): The prefix of info filenames.
        out_dir (str): Output directory of the generated info file.
        workers (int): Number of threads to be used.
        max_sweeps (int, optional): Number of input consecutive frames.
            Default: 5. Here we store pose information of these frames
            for later use.
    """
    from tools.dataset_converters import waymo_converter as waymo

    splits = [
        'training', 'validation', 'testing', 'testing_3d_camera_only_detection'
    ]
    for i, split in enumerate(splits):
        load_dir = osp.join(root_path, 'waymo_format', split)
        if split == 'validation':
            save_dir = osp.join(out_dir, 'kitti_format', 'training')
        else:
            save_dir = osp.join(out_dir, 'kitti_format', split)
        converter = waymo.Waymo2KITTI(
            load_dir,
            save_dir,
            prefix=str(i),
            workers=workers,
            test_mode=(split
                       in ['testing', 'testing_3d_camera_only_detection']))
        converter.convert()

    from tools.dataset_converters.waymo_converter import \
        create_ImageSets_img_ids
    create_ImageSets_img_ids(osp.join(out_dir, 'kitti_format'), splits)
    # Generate waymo infos
    out_dir = osp.join(out_dir, 'kitti_format')
    kitti.create_waymo_info_file(
        out_dir, info_prefix, max_sweeps=max_sweeps, workers=workers)
    info_train_path = osp.join(out_dir, f'{info_prefix}_infos_train.pkl')
    info_val_path = osp.join(out_dir, f'{info_prefix}_infos_val.pkl')
    info_trainval_path = osp.join(out_dir, f'{info_prefix}_infos_trainval.pkl')
    info_test_path = osp.join(out_dir, f'{info_prefix}_infos_test.pkl')
    update_pkl_infos('waymo', out_dir=out_dir, pkl_path=info_train_path)
    update_pkl_infos('waymo', out_dir=out_dir, pkl_path=info_val_path)
    update_pkl_infos('waymo', out_dir=out_dir, pkl_path=info_trainval_path)
    update_pkl_infos('waymo', out_dir=out_dir, pkl_path=info_test_path)
    GTDatabaseCreater(
        'WaymoDataset',
        out_dir,
        info_prefix,
        f'{info_prefix}_infos_train.pkl',
        relative_path=False,
        with_mask=False,
        num_worker=workers).create()

def semantickitti_data_prep(info_prefix, out_dir):
    """Prepare the info file for SemanticKITTI dataset.

    Args:
        info_prefix (str): The prefix of info filenames.
        out_dir (str): Output directory of the generated info file.
    """
    semantickitti_converter.create_semantickitti_info_file(
        info_prefix, out_dir)

parser = argparse.ArgumentParser(description='Data converter arg parser')
parser.add_argument('dataset', metavar='kitti', help='name of the dataset')
parser.add_argument(
    '--root-path',
    type=str,
    default='./data/kitti',
    help='specify the root path of dataset')
parser.add_argument(
    '--version',
    type=str,
    default='v1.0',
    required=False,
    help='specify the dataset version, no need for kitti')
parser.add_argument(
    '--max-sweeps',
    type=int,
    default=10,
    required=False,
    help='specify sweeps of lidar per example')
parser.add_argument(
    '--with-plane',
    action='store_true',
    help='Whether to use plane information for kitti.')
parser.add_argument(
    '--out-dir',
    type=str,
    default='./data/kitti',
    required=False,
    help='name of info pkl')
parser.add_argument('--extra-tag', type=str, default='kitti')
parser.add_argument(
    '--workers', type=int, default=4, help='number of threads to be used')
parser.add_argument(
    '--only-gt-database',
    action='store_true',
    help='Whether to only generate ground truth database.')
args = parser.parse_args()

if __name__ == '__main__':
    from mmdet3d.utils import register_all_modules
    register_all_modules()

    if args.dataset == 'kitti':
        if args.only_gt_database:
            create_groundtruth_database(
                'KittiDataset',
                args.root_path,
                args.extra_tag,
                f'{args.extra_tag}_infos_train.pkl',
                relative_path=False,
                mask_anno_path='instances_train.json',
                with_mask=(args.version == 'mask'))
        else:
            kitti_data_prep(
                root_path=args.root_path,
                info_prefix=args.extra_tag,
                version=args.version,
                out_dir=args.out_dir,
                with_plane=args.with_plane)
    elif args.dataset == 'nuscenes' and args.version != 'v1.0-mini':
        if args.only_gt_database:
            create_groundtruth_database('NuScenesDataset', args.root_path,
                                        args.extra_tag,
                                        f'{args.extra_tag}_infos_train.pkl')
        else:
            train_version = f'{args.version}-trainval'
            nuscenes_data_prep(
                root_path=args.root_path,
                info_prefix=args.extra_tag,
                version=train_version,
                dataset_name='NuScenesDataset',
                out_dir=args.out_dir,
                max_sweeps=args.max_sweeps)
            test_version = f'{args.version}-test'
            nuscenes_data_prep(
                root_path=args.root_path,
                info_prefix=args.extra_tag,
                version=test_version,
                dataset_name='NuScenesDataset',
                out_dir=args.out_dir,
                max_sweeps=args.max_sweeps)
    elif args.dataset == 'nuscenes' and args.version == 'v1.0-mini':
        if args.only_gt_database:
            create_groundtruth_database('NuScenesDataset', args.root_path,
                                        args.extra_tag,
                                        f'{args.extra_tag}_infos_train.pkl')
        else:
            train_version = f'{args.version}'
            nuscenes_data_prep(
                root_path=args.root_path,
                info_prefix=args.extra_tag,
                version=train_version,
                dataset_name='NuScenesDataset',
                out_dir=args.out_dir,
                max_sweeps=args.max_sweeps)
    elif args.dataset == 'lyft':
        train_version = f'{args.version}-train'
        lyft_data_prep(
            root_path=args.root_path,
            info_prefix=args.extra_tag,
            version=train_version,
            max_sweeps=args.max_sweeps)
        test_version = f'{args.version}-test'
        lyft_data_prep(
            root_path=args.root_path,
            info_prefix=args.extra_tag,
            version=test_version,
            max_sweeps=args.max_sweeps)
    elif args.dataset == 'waymo':
        waymo_data_prep(
            root_path=args.root_path,
            info_prefix=args.extra_tag,
            version=args.version,
            out_dir=args.out_dir,
            workers=args.workers,
            max_sweeps=args.max_sweeps)
    elif args.dataset == 'scannet':
        scannet_data_prep(
            root_path=args.root_path,
            info_prefix=args.extra_tag,
            out_dir=args.out_dir,
            workers=args.workers)
    elif args.dataset == 's3dis':
        s3dis_data_prep(
            root_path=args.root_path,
            info_prefix=args.extra_tag,
            out_dir=args.out_dir,
            workers=args.workers)
    elif args.dataset == 'sunrgbd':
        sunrgbd_data_prep(
            root_path=args.root_path,
            info_prefix=args.extra_tag,
            out_dir=args.out_dir,
            workers=args.workers)
    elif args.dataset == 'semantickitti':
        semantickitti_data_prep(
            info_prefix=args.extra_tag, out_dir=args.out_dir)
    else:
        raise NotImplementedError(f'Don\'t support {args.dataset} dataset.')

Reproduces the problem - command or script

python tools/create_data.py custom --root-path ./data/custom --out-dir ./data/custom --extra-tag custom

Reproduces the problem - error message

def get_thresholds(scores: np.ndarray, num_gt, num_sample_pts=41): Traceback (most recent call last): File "tools/create_data.py", line 380, in raise NotImplementedError(f'Don\'t support {args.dataset} dataset.') NotImplementedError: Don't support custom dataset.

Additional information

As you can see in the image above, you also mention it in the documentation section([]https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_dataset.html)). We can create a custom dataset with a data structure like the image above. But because there is no "custom" tag in the tools/create_data.py folder, I get the error "NotImplementedError: Don't support custom dataset". What should I do? Can you help me?

[JingweiZhang12]

If you want to customize your dataset, you should add your preprocess code and modify the choices of argument dataset

[kbratsy]

If you want to customize your dataset, you should add your preprocess code and modify the choices of argument dataset

Could you give some more detailed information? Because when I tried to follow the steps in the parts I shared in the image below, I encountered the problem above. Link: (https://mmdetection3d.readthedocs.io/en/latest/)

[reyrg2021]

I have the same problem, can you help us?

[HydrogenWasser]

if u want to use custom dataset u have to write a new data_converter and pkl_maker, the converter in mm3d only suits for standard datasets

[Agbeli]

Hi @HydrogenWasser can you share a sample code? I am facing the same issue here. Thank you

[HydrogenWasser]

Hi @HydrogenWasser can you share a sample code? I am facing the same issue here. Thank you

share ur email

[MarvinKlemp]

@HydrogenWasser running into the same issue.

I also don't really understand whats the point of the documentation. If you have a standardized custom format I assume it's also possible to provide a data_converter for it :(

Anyways, it would help me quite a lot if you could also share your impl. marvin.klemp@kit.edu

[kbratsy]

Hi @HydrogenWasser can you share a sample code? I am facing the same issue here. Thank you

share ur email

Can you share it with me also? atasoy.kubraa@hotmail.com

[Agbeli]

Hi @HydrogenWasser can you share a sample code? I am facing the same issue here. Thank you

share ur email

agbeliemmanuel@gmail.com

[Agbeli]

Thank you for your assistance. This would be helpful to me. I think they should update the documentation when it comes to custom datasets.

[reyrg2021]

Hi @HydrogenWasser can you share a sample code? I am facing the same issue here. Thank you

reinierrg.2020@gmail.com

[whuyyc]

@HydrogenWasser Hi. can you share a sample code? I am facing the same issue here. Thank you very much! 970506520@qq.com

[PuChan-HCI]

@HydrogenWasser, would be grateful if you can share a sample code to me. o.d.a.prima@gmail.com Thank you in advance.

[guavamin]

@HydrogenWasser I'm running into the same question. This is my email kengmin8933@gmail.com. Thanks in advance!

[man-droid23]

Hi @HydrogenWasser .Can you share the sample code? I am facing the same issue here. Thank in advance! jonijojoz23@gmail.com

[isco-a]

Hi @HydrogenWasser . Could you please share the sample code? dwcha5015@gmail.com Thanks in advance!

[hanlinhung]

Hi @HydrogenWasser , I'm running into the same question, could you share the sample code? hanlinhung@gmail.com Tranks in advance.

[andrewcwlee]

@Agbeli @MarvinKlemp @kbratsy @HydrogenWasser Could you share the sample code? itisandrewlee@gmail.com This would be a great help. Thanks in advance.

[9Alex2]

Hi @HydrogenWasser can you share a sample code? I am facing the same issue here. Thank you

share ur email

Your code would be a great help for me, too. Thanks a lot in advance! alex-luna@gmx.de

[lxzjlu]

Hi，Could you share the sample code? It would be very useful to me. xz15209852118@163.com Thanks you very much!!! @HydrogenWasser @kbratsy @Agbeli

[linClubs]

Hi @HydrogenWasser . Could you please share the sample code? yonglinh611@gmail.com Thanks in advance!

[hrysj2016]

Hi @HydrogenWasser . Could you please share the sample code? 12221089@zju.edu.cn Thanks in advance!

[MartinaAnn]

Hi @HydrogenWasser I've got the same problem while developing my master thesis, could you share the sample code with me too? martina.annicelli@gmail.com Thanks in advance

[imbama]

Hi @HydrogenWasser . Could you please share the sample code? 23121778@bjtu.edu.cn. Thanks in advance!

[JosephThomas007]

Hi @HydrogenWasser . Could you please share the sample code to me as well? I am struggling to create a custom dataset. joseph.sjcetmech@gmail.com

[luvubnd]

Hi, @HydrogenWasser @Agbeli .Could you please share the sample code to me as well? It would be extremely helpful. 62990023@kmitl.ac.th Thanks in advance!

[cwx-cv]

Hi @HydrogenWasser . Could you please share the sample code to me as well? I am struggling to create a custom dataset. 1104461939@qq.com

[MarvinKlemp]

I cost me some time, but I have implemented the training for pointcloud only datasets and could easily manage to create a PR and some docs which would reduce the effort to implement this task to hours.

However, I am still struggling with two things @JingweiZhang12 : First: For what is the gt directory at all? Why is it created? Is it only for the ObjectSampler? Second: #2242 I am unable to get the evaluation running at all because the KITTI Evaluator requires images. And I don't really understand how this was solved in the issue @m-shahbaz-kharal maybe you could help?

[m-shahbaz-kharal]

I cost me some time, but I have implemented the training for pointcloud only datasets and could easily manage to create a PR and some docs which would reduce the effort to implement this task to hours.

However, I am still struggling with two things @JingweiZhang12 : First: For what is the gt directory at all? Why is it created? Is it only for the ObjectSampler? Second: #2242 I am unable to get the evaluation running at all because the KITTI Evaluator requires images. And I don't really understand how this was solved in the issue @m-shahbaz-kharal maybe you could help?

@MarvinKlemp, gt directory is generated as sort of an efficiency procedure, it is generated by cropping objects of interest (for example vehicles, pedestrians, cyclists, etc.) from point-cloud frames and storing them (the cropped point-clouds containing individual objects of interest). This is done utilizing the ground-truth labels. Once gt directory is created it is easy to compare the detections to ground-truth objects individually to calculate the scoring metric (mAP).

Regarding KITTI Evaluator, I wasn't able to get results. If you are working on point-cloud only dataset I'd suggest you use OpenPCDet. That is more clean and easy to understand library from open-mmlab.

[MarvinKlemp]

@m-shahbaz-kharal thanks for the fast answer.

About the gt dir I'm not sure if it is really used in validation. Because from what I can see its only used for the ObjectSampler in training

About the Evaluator, that's sad to hear. I spend yesterday to try to get it running and have a "running" version. But it feels hacked. The one used in OpenPCDet also seems to be quite hard to use and buggy. Furthermore it also requires the 2D Information.

However, I think that the Waymo Evaluation only uses 3D information, hence it might be possible to easily generate gt.bin/pred.bin files from your labels and predictions. I will test that out.

[VeeranjaneyuluToka]

GT dir is not being used in validation rather it is being used in training itself. We can train a model without GT dir as well. GT dir is basically some sampling techniques and filtering is getting applied on the input data esp on point clouds. I mostly adopted NuScenes format for both .pkl generation and evaluation code and it works fine.

[SimonTUBS]

I cost me some time, but I have implemented the training for pointcloud only datasets and could easily manage to create a PR and some docs which would reduce the effort to implement this task to hours.

However, I am still struggling with two things @JingweiZhang12 : First: For what is the gt directory at all? Why is it created? Is it only for the ObjectSampler? Second: #2242 I am unable to get the evaluation running at all because the KITTI Evaluator requires images. And I don't really understand how this was solved in the issue @m-shahbaz-kharal maybe you could help?

@MarvinKlemp Do you mind sharing some insights. I also want to implement training for pointcloud only datasets.

[PraveenKumar-Rajendran]

if u want to use custom dataset u have to write a new data_converter and pkl_maker, the converter in mm3d only suits for standard datasets

Hi @HydrogenWasser @MarvinKlemp @VeeranjaneyuluToka

Could you please share your insights on data preparation? I understand that the 'custom' tag is not implemented in create_data.py. This is confusing because the documentation refers to it as if it were already implemented.

Additionally, what is described as 'basic data format' in the documentation does not exactly match the KITTI format when you examine the label format.

Given this, I assume we need to convert our data to a standard format like KITTI. FYI, I work with image-only datasets.

Thank you for your time.

[VeeranjaneyuluToka]

@PraveenKumar-Rajendran , Below are the steps that i followed to adopt to custom dataset Step1: Implement custom dataset class which is derived from Det3DDataset in mmdet3d->datasets folder Step2: Create a new config based on the architecture that we choose (for example centerpoint) and update dataset_type with the above custom data class name Step3: Create data converter by following either Kitti or NuScenes format. I followed NeScenes format. I have done lots of modifications to just populate the required dict format to forward function. This is completely based on this git repo (https://github.com/mit-han-lab/bevfusion/tree/main/tools/data_converter)

The above three steps should allow you train model successfully. You need to follow another step to evaluate the model.

Step4: Adopt to either kitti or NuScenes evaluation metric to evaluate your model. I followed NuScenes evaluation based on (https://github.com/nutonomy/nuscenes-devkit) and modified as i need.

Hope this will help you, let me know if you need any more details. Thanks!