search

autonomousvision / kitti360Scripts

This repository contains utility scripts for the KITTI-360 dataset.
MIT License
379 stars 61 forks source link

json label support for panoptic training #5

Open griffintin opened 3 years ago

griffintin commented 3 years ago

Hi @yiyiliao , Thank you for your great work.

Will the json label of instance be available?

In the folder tree of http://www.cvlibs.net/datasets/kitti-360/documentation.php, instanceDict.json appears, but no in the downloaded data.
If I want to train for panoptic segmentation, how can I generate the training labels?

yiyiliao commented 3 years ago

Hi @griffintin , thanks for your question. You can find instanceDist.json here: image

The 2D instance segmentation masks can be considered as labels for panoptic segmentation, where every pixel is labeled in the form of semanticId*1000 + instanceId.

griffintin commented 3 years ago

Hi @yiyiliao

Thank you for your quick reply. I did not find .json file as you mentioned in "3D bounding box" data. Anyway, I made instance.json from .png files from data_2d_semantics/instance.

And I found another problem. For every sequence, for part of RGB images in data_2d_raw, there is no corresponding instance .png in data_2d_semantics. If so, I think unnecessary ones better to be deleted. currenly, RGB images is about 83000 but instance is less than that.

pwais commented 3 years ago

@griffintin the "3d bounding box" data is in the data_3d_bboxes download and is not in JSON format. See this issue for some context that might be helpful: https://github.com/autonomousvision/kitti360Scripts/issues/6#issuecomment-729262235

yiyiliao commented 3 years ago

Hi @griffintin ,

Thank you for your quick reply. I did not find .json file as you mentioned in "3D bounding box" data. Anyway, I made instance.json from .png files from data_2d_semantics/instance.

Sorry I misunderstood your question. We do not provide instance.json per frame for panoptic segmentation. Good that you made them yourself!

And I found another problem. For every sequence, for part of RGB images in data_2d_raw, there is no corresponding instance .png in data_2d_semantics. If so, I think unnecessary ones better to be deleted. currenly, RGB images is about 83000 but instance is less than that.

This is true. We have some held-out instance labels that we plan to use for our benchmark while the corresponding RGB images have already been released.

MichelleSchaaf commented 2 years ago

Hi @yiyiliao

Thank you for your quick reply. I did not find .json file as you mentioned in "3D bounding box" data. Anyway, I made instance.json from .png files from data_2d_semantics/instance.

Hey, I am also trying to obtain an instance.json for training instance segmentation. However, when trying to collect binary masks for every single instance of an image, I have the problem that multiple objects in one image have the same label.

For example instance_segmentation 2013_05_28_drive_0000_sync/0000000253.png. There are multiple cars with the same instance label 101: grafik

What am I missing? How can I get a binary mask of just one object and not the whole class?

Thank you in advance for any help!

yiyiliao commented 2 years ago

Hi @MichelleSchaaf, sorry for the late reply due to the summer break. Is there any chance that you did not load the instance mask in the format of uint16? Please refer to our documentation page:

Instance label in single-channel 16-bit PNG format. Each pixel value denotes the corresponding instanceID. Here, instanceID = semanticID*1000 + classInstanceID with classInstanceID denoting the instance ID within one class and classInstanceID = 0 for classes without instance label.