Waymo Dataset converter.py

[mingyuShin]

First of all, thank you for the great work. I used the converter as instructed to convert the Waymo dataset. Afterward, I counted the number of images and calibrations in the Waymo validation_org set and found that there were 39,987 of each, but only 39,047 labels. When I applied the setup_split, I discovered that some data had not been converted due to the lack of labels. As a result, I only have 51,257 training samples and 38,960 validation samples, while your paper states that there are 52,386 training and 39,848 validation samples. How can I obtain the same number of samples as described in the paper (52,386, 39,848)?

[abhi1kumar]

Hi @mingyuShin Thank you for your interest in DEVIANT. I quickly checked my waymo validation_org and validation folders. My validation contains 39,848 images we report in the paper. Here is a screenshot of the same:

Your number of validation_org labels is different, which results in lower images for the validation set after setup_split.py. Could you please let me know which images of this ImageSet are absent in validation_org folder during conversion?

I could paste their labels here to confirm if they are empty images.

[mingyuShin]

empty_val.txt empty_trian.txt @abhi1kumar, thanks for your help. I don't have these labels in both train and val.

[abhi1kumar]

Not all of the training are empty files and therefore, I upload the calib and label subfolders of the waymo training and validation split at this drive link.

Unzip the above file and place them as follows:

./code
├── data
│      └── waymo
│             ├── ImageSets
│             ├── training
│             │     ├── calib
│             │     └── label
│             │
│             └── validation
│                   ├── calib
│                   └── label

Then consider copying the corresponding images in the image sub-folder of training and validation folders to complete the folder structure:

./code
├── data
│      └── waymo
│             ├── ImageSets
│             ├── training
│             │     ├── calib
│             │     ├── image
│             │     └── label
│             │
│             └── validation
│                   ├── calib
│                   ├── image
│                   └── label

Let me know if that helps.

[mingyuShin]

You are a really kind person. Thank you so much for the helpful explanation. The issue has been resolved.