How to use it in kitti?

[Abyssaledge]

Our implementation is based on MMDetection3D. It's easy to use SST in KITTI with their instructions. I believe you just need to prepare dataset and make minor modifications to the config file. You can find these modifications by comparing the following two configs. https://github.com/open-mmlab/mmdetection3d/blob/master/configs/_base_/models/hv_pointpillars_secfpn_waymo.py https://github.com/open-mmlab/mmdetection3d/blob/master/configs/_base_/models/hv_pointpillars_secfpn_kitti.py Instruction of KITTI in MMDetection3D: https://github.com/open-mmlab/mmdetection3d/blob/master/docs/zh_cn/datasets/kitti_det.md

[Tongjiaxun]

KeyError: "DynamicVoxelNet: 'SSTv1 is not in the models register' "

Do you know how to resolve this error?

Thank you very much!

[Abyssaledge]

Do you modify any codes? It seems that you do not register the SSTv1 like: https://github.com/TuSimple/SST/blob/main/mmdet3d/models/backbones/sst_v1.py#L17

[Tongjiaxun]

I did not modify the codes ,I think the problem is from my mmdet3d package , my enviroment can work in mmdetction3d . But the mmdet3d package in SST is different from the one in mmdedection3d , maybe some files missing such as 'ball_query_ext.cpython-37m-x86_64-linux-gnu.so', I do not have these files in mmdet3d package.

[Abyssaledge]

I guess you have the official MMDetection3D installed, but you do not install SST. You need first uninstall the official mmdet3d by pip uninstall mmdet3d and install SST by running pip install -v -e . under the SST directory.

[Tongjiaxun]

When I use pip intsall -v -e . in SST have error Using pip 21.2.2 from /home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pip (python 3.7) Obtaining file:///home/bai/Documents/Deeplearning/SST Running command python setup.py egg_info Traceback (most recent call last): File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 98, in init req = REQUIREMENT.parseString(requirement_string) File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 1654, in parseString raise exc File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 1644, in parseString loc, tokens = self._parse( instring, 0 ) File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache loc,tokens = self.parseImpl( instring, preloc, doActions ) File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 3417, in parseImpl loc, exprtokens = e._parse( instring, loc, doActions ) File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache loc,tokens = self.parseImpl( instring, preloc, doActions ) File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 3739, in parseImpl return self.expr._parse( instring, loc, doActions, callPreParse=False ) File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 1402, in _parseNoCache loc,tokens = self.parseImpl( instring, preloc, doActions ) File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 3400, in parseImpl loc, resultlist = self.exprs[0]._parse( instring, loc, doActions, callPreParse=False ) File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 1406, in _parseNoCache loc,tokens = self.parseImpl( instring, preloc, doActions ) File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pkg_resources/_vendor/pyparsing.py", line 2711, in parseImpl raise ParseException(instring, loc, self.errmsg, self) pkg_resources._vendor.pyparsing.ParseException: Expected W:(abcd...) (at char 0), (line:1, col:1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/bai/Documents/Deeplearning/SST/setup.py", line 259, in <module>
    zip_safe=False)
  File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/setuptools/__init__.py", line 152, in setup
    _install_setup_requires(attrs)
  File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/setuptools/__init__.py", line 147, in _install_setup_requires
    dist.fetch_build_eggs(dist.setup_requires)
  File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/setuptools/dist.py", line 809, in fetch_build_eggs
    replace_conflicting=True,
  File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pkg_resources/__init__.py", line 728, in resolve
    requirements = list(requirements)[::-1]
  File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3084, in parse_requirements
    yield Requirement(line)
  File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3094, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 102, in __init__
    requirement_string[e.loc : e.loc + 8], e.msg
pkg_resources.extern.packaging.requirements.InvalidRequirement: Parse error at "'\x00\x00\x00\x00\x00\x00\x00\x00'": Expected W:(abcd...)

WARNING: Discarding file:///home/bai/Documents/Deeplearning/SST. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output. ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

I do not know the reason.

[Abyssaledge]

There should not be any problems if you can install MMDetection3D successfully. Because this project is purely based on MMDetection3D and we do not modify the setup process. Have you ever tried to install MMDetection3D as the way you install SST?

[Tongjiaxun]

Yes , it works and have a new error

Traceback (most recent call last): File "tools/train.py", line 16, in from mmdet3d.apis import train_model File "/home/bai/Documents/Deeplearning/SST/mmdet3d/apis/init.py", line 1, in from .inference import (convert_SyncBN, inference_detector, File "/home/bai/Documents/Deeplearning/SST/mmdet3d/apis/inference.py", line 16, in from mmdet3d.models import build_model File "/home/bai/Documents/Deeplearning/SST/mmdet3d/models/init.py", line 1, in from .backbones import * # noqa: F401,F403 File "/home/bai/Documents/Deeplearning/SST/mmdet3d/models/backbones/init.py", line 7, in from .sst_v1 import SSTv1 File "/home/bai/Documents/Deeplearning/SST/mmdet3d/models/backbones/sst_v1.py", line 7, in from mmdet3d.models.sst.sst_basic_block import BasicShiftBlock ValueError: source code string cannot contain null bytes

maybe the reason is my file is windows to linux

[Abyssaledge]

Yes, It looks like some text encoding errors. Although I have no specific experience with this, it might work if you directly clone this repo to your local file system. Let me know if you have any other problems related to SST.

[Tongjiaxun]

That error has been resolved and this is another error when I use "python tools/train.py configs/sst/sst_waymoD1_2x_3class_8heads_3f.py " to train.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "tools/train.py", line 230, in main() File "tools/train.py", line 194, in main datasets = [build_dataset(cfg.data.train)] File "/home/bai/Downloads/SST/mmdet3d/datasets/builder.py", line 30, in build_dataset build_dataset(cfg['dataset'], default_args), cfg['times']) File "/home/bai/Downloads/SST/mmdet3d/datasets/builder.py", line 39, in build_dataset dataset = build_from_cfg(cfg, DATASETS, default_args) File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/mmcv/utils/registry.py", line 54, in build_from_cfg raise type(e)(f'{obj_cls.name}: {e}') TypeError: KittiDataset: init() got an unexpected keyword argument 'load_interval'

I have modified sst_waymoD1_2x_3class_8heads_3f.py

base = [ '../base/models/sst_base.py',

'../base/datasets/waymo-3d-3class-3sweep.py',

'../_base_/datasets/kitti-3d-3class.py',
'../_base_/schedules/cosine_2x.py',
'../_base_/default_runtime.py',

]

[Abyssaledge]

You may not be able to switch to KITTI with such a simple substitution. Instead, I suggest you go through the whole config to find out the related items. At least, point_cloud_range, anchor-generator, and assigner should be changed. Please refer to https://github.com/open-mmlab/mmdetection3d/blob/master/configs/_base_/models/hv_second_secfpn_kitti.py

The data item in sst_waymoD1_2x_3class_8heads_3f override the default data in kitti-3d-3class, so you got the error. Some documents in MMDetection3D might be helpful to you.

[Tongjiaxun]

I have modified these parameters but it still can not work, are there any ways to check these parameters?

[Abyssaledge]

After starting running an experiment. there is an automatically generated config file in work_dirs. So you can run two experiments with the same model in MMDetection3D, one for KITTI and one for Waymo. Then you can compare the two generated configs to decide which part needs modification. Tips: it's ok if you do not prepare waymo data, because you only need the automatically generated config.

[Tongjiaxun]

middle_encoder=dict(
    type='SSTInputLayer',
    window_shape=window_shape,
    shifts_list=shifts_list,
    point_cloud_range=point_cloud_range,
    voxel_size=voxel_size,
    shuffle_voxels=True,
    debug=True,
    drop_info=drop_info,
),   

neck=dict( type='SECONDFPN',

norm_cfg=dict(type='naiveSyncBN2d', eps=1e-3, momentum=0.01),

    in_channels=[128,],
    upsample_strides=[1,],
    out_channels=[384, ]
),
backbone=dict(
    type='SSTv1',
    d_model=[128,] * 6,
    nhead=[8, ] * 6,
    num_blocks=6,
    dim_feedforward=[256, ] * 6,
    output_shape=[468, 468],
    num_attached_conv=3,
    conv_kwargs=[
        dict(kernel_size=3, dilation=1, padding=1, stride=1),
        dict(kernel_size=3, dilation=1, padding=1, stride=1),
        dict(kernel_size=3, dilation=2, padding=2, stride=1),
    ],
    conv_in_channel=128,
    conv_out_channel=128,
    debug=True,
    drop_info=drop_info,
    pos_temperature=10000,
    normalize_pos=False,
    window_shape=window_shape,
    checkpoint_blocks=[0,1,2,3]
),

)

Are there anything need to be modified in here ?

[Abyssaledge]

point_cloud_range and output_shape should be modified for KITTI.

[Tongjiaxun]

I think I have modified all the parameters need to be done , but it still can not work . Thank you very much for teaching me!

[Abyssaledge]

What do you mean by "can not work"? What is the specific error?

[Tongjiaxun]

Traceback (most recent call last): File "tools/train.py", line 230, in main() File "tools/train.py", line 194, in main datasets = [build_dataset(cfg.data.train)] File "/home/bai/Downloads/SST/mmdet3d/datasets/builder.py", line 30, in build_dataset build_dataset(cfg['dataset'], default_args), cfg['times']) File "/home/bai/Downloads/SST/mmdet3d/datasets/builder.py", line 39, in build_dataset dataset = build_from_cfg(cfg, DATASETS, default_args) File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/mmcv/utils/registry.py", line 54, in build_from_cfg raise type(e)(f'{obj_cls.name}: {e}') TypeError: KittiDataset: init() got an unexpected keyword argument 'load_interval' I have modified many parameters. Can I add your Wechat? I have something want to ask you. My phone number is 18275148115

[SH-Tan]

Traceback (most recent call last): File "tools/train.py", line 230, in main() File "tools/train.py", line 194, in main datasets = [build_dataset(cfg.data.train)] File "/home/bai/Downloads/SST/mmdet3d/datasets/builder.py", line 30, in build_dataset build_dataset(cfg['dataset'], default_args), cfg['times']) File "/home/bai/Downloads/SST/mmdet3d/datasets/builder.py", line 39, in build_dataset dataset = build_from_cfg(cfg, DATASETS, default_args) File "/home/bai/anaconda3/envs/mmdetection3d/lib/python3.7/site-packages/mmcv/utils/registry.py", line 54, in build_from_cfg raise type(e)(f'{obj_cls.name}: {e}') TypeError: KittiDataset: init() got an unexpected keyword argument 'load_interval' I have modified many parameters. Can I add your Wechat? I have something want to ask you. My phone number is 18275148115

Hi, I am also trying to train SST using Kitti, and there are some problems, do you run the model successfully, can I refr to you config file?