ERR : Generate GT database

[git-sachin1066]

Thanks for uploading code and weight, i have trouble in understanding Generate GT database step mentioned as _python -m pcdet.datasets.kitti.sem_painted_kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/semantic_paintedkitti.yaml

what is thses values 1) pcdet.datasets.kitti.sem_painted_kitti_dataset 2) create_kitti_infos Any idea where to get those
thanks for help

[HAMA-DL-dev]

Oops, I forget to upload sem_painted_kitti_dataset.py.

This file will be uploaded if there is any permission to share it from reviewers.

Anyway, let me explain those step by step.

1. pcdet.datasets.kitti.sem_painted_kitti_dataset

Purpose This is config file for "how to generate GT base". It dependes on what, how to generate GT database.

And the database is used for training, including data augmentation in our paper (sorry for the paper is not available due to the review process and double bline submission)

"So what is GT database is for?", this is important question. This is label data to train the model.

After executing pcdet.datasets.kitti.sem_painted.kitti_dataset, you can get kitti_infos_train.pkl, kitti_infos_val.pkl, and kitti_dbinfos_train.pkl.

Value sem_painted_kitti_dataset is a name of executable python file in located

/path/to/your/SeSame/detector/pcdet/datasets/kitti/
    ├── kitti_dataset.py
    ├── kitti_object_eval_python
    ├── kitti_utils.py
    ├── painted_kitti_dataset.py
    └── **sem_painted_kitti_dataset.py**   # <---- I forgot to upload this one. T.T so you would get the error

Source : To understand how those work, followings will be helpful to get it.

• Codes

  • pcdet/datsets/kitti/kitti_dataset.py <--- our code is designed based on this
  • pcdet/datasets/augmentor/data_augmentor.py <--- this file includes the data augmentation

• Link

  • mmdet3d : "how to create kitti dataset infos?" (link) I highly recommend to to read this document. This docs explain briefly what consists of the GT database file.

In conclusion, pcdet.datasets.kitti.sem_painted_kitti_dataset is

A file specified what, where, how to create GT database(~= label data), including data augmentation

2. create_kitti_infos

If you understand above one. This is just simple.

In the file named sem_painted_kitti_dataset.py,

if __name__ == '__main__':
    import sys
    if sys.argv.__len__() > 1 and sys.argv[1] == 'create_kitti_infos':  #   !<------ This line!
        import yaml
        from pathlib import Path
        from easydict import EasyDict
        dataset_cfg = EasyDict(yaml.full_load(open(sys.argv[2])))
        ROOT_DIR = Path(dataset_cfg['DATA_PATH']).resolve()
        print(ROOT_DIR)
        create_kitti_infos(   #   !<------ This is the main function of generating the GT database                                   
            dataset_cfg=dataset_cfg,
            class_names=['Car', 'Pedestrian', 'Cyclist'],
            data_path=ROOT_DIR,
            save_path=ROOT_DIR
        )

This is just an argument passed through the python script.

Generally, pepople use argparser to get the argument, but we use sys.argv

Why is there an error?

As I mentioned above, I forgot to upload sem_painted_kitti_dataset.py.

If you have the file and data path is correct, the error will be solved.

I apologize if my explanation was not entirely clear due to my limited English proficiency. Please feel free to ask any questions at your convenience, as I am available to provide prompt responses.

Thank you for your interest in my work ( ͡° ͜ʖ ͡°)

[a-free-a]

pcdet.datasets.kitti.sem_painted.kitti_dataset

Dear author, I tried to create Generate GT database by myself, but it didn't work, can you share a copy of sem_painted_kitti_dataset.py code, if it's convenient for you, my email address is 221068407@stdmail.gxust.edu.cn

[HAMA-DL-dev]

pcdet.datasets.kitti.sem_painted.kitti_dataset

Dear author, I tried to create Generate GT database by myself, but it didn't work, can you share a copy of sem_painted_kitti_dataset.py code, if it's convenient for you, my email address is 221068407@stdmail.gxust.edu.cn

I sent an mail attached the file.

Thanks for your interest in my work :3

[HAMA-DL-dev]

Thanks for uploading code and weight, i have trouble in understanding Generate GT database step mentioned as _python -m pcdet.datasets.kitti.sem_painted_kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/semantic_paintedkitti.yaml

what is thses values

1. pcdet.datasets.kitti.sem_painted_kitti_dataset
2. create_kitti_infos Any idea where to get those thanks for help

As specified in README.md, the infos for KITTI are generated via the command below

cd /path/to/your/SeSame/detector/
python -m pcdet.datasets.kitti.sem_painted_kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/semantic_painted_kitti.yaml

Expected Erorr 1 : no lidar_file with extension *.npy

However, below one can be a solution for expected error with

File "/path/to/your/SeSame/detector/pcdet/datasets/kitti/painted_kitti_dataset.py", line 74, in get_painted_lidar
    assert lidar_file.exists(),f"{lidar_file}"
AssertionError: /path/to/your/Kitti/training/painted_lidar/*.npy

If you can understand the comment, you might get a question ; the name of the file is wrong, isn't it?

The question is right. So the error occurs.

You can modify ~/SeSame/detector/pcdet/datasets/__init__.py just like below

import torch
from torch.utils.data import DataLoader
from torch.utils.data import DistributedSampler as _DistributedSampler

from pcdet.utils import common_utils

from .dataset import DatasetTemplate
from .kitti.kitti_dataset import KittiDataset
############### This line! ###############
from .kitti.sem_painted_kitti_dataset import PaintedKittiDataset
#####################################
from .nuscenes.nuscenes_dataset import NuScenesDataset
from .waymo.waymo_dataset import WaymoDataset

__all__ = {
    'DatasetTemplate': DatasetTemplate,
    'KittiDataset': KittiDataset,
############### This line! ###############
    'PaintedKittiDataset': PaintedKittiDataset,
#####################################
    'NuScenesDataset': NuScenesDataset,
    'WaymoDataset': WaymoDataset
}

I have checked the file in this repository DOES NOT modified like above.

This might be helpful.

[shuxuejianmo1]

Thank you for sharing. I am very interested in your work . If it's convenient, could you send me a copy of the "sem_painted_kitti_dataset.py" code? My email address is 202211901015@stu.hebut.edu.cn，best wishes！

[HAMA-DL-dev]

Thank you for sharing. I am very interested in your work . If it's convenient, could you send me a copy of the "sem_painted_kitti_dataset.py" code? My email address is 202211901015@stu.hebut.edu.cn，best wishes！

Of course YES. I sent the email attached sem_painted_kitti_dataset.py

And please refer above comments (this, and this) for expected errors.

Thanks for your interest in my work ( ᐛ )