search

waymo-research / waymo-open-dataset

Waymo Open Dataset
https://www.waymo.com/open
Other
2.73k stars 617 forks source link

2D semmantic error #634

Open swimmerQAQ opened 1 year ago

swimmerQAQ commented 1 year ago

Sir, I meet an error that I check some semantic label images, which I extract from tfrecord file. It seems like the image have some area of sky, while its label doesn't have label "sky : 25", and the sky area is label with "car : 2" this kind. label map to cityscapes and visualization 1681285381(1)

other examples origin label and after map to cityscapes 1681285435(1) origin imgae waymo_full_val_scene12_11_leftImg8bit

swimmerQAQ commented 1 year ago

I guess the waymo semantic labels have different versions, which I don't know. Leads to different definition. My definition is: 1681285778(1) 1681285800(1)

alexzzhu commented 1 year ago

Is this an issue that you're seeing in every frame, or just this one frame? Our class spec is encoded as a proto here: https://github.com/waymo-research/waymo-open-dataset/blob/bd88b99f2e2de552f66b082e58a434c154bb28ae/src/waymo_open_dataset/protos/camera_segmentation.proto, which seems to match your definition.

swimmerQAQ commented 1 year ago

Oh, thanks for your timely reply! Yes, this definition is what I refer to. I just find is in one tfrecord file, not many frames error. I didn't calculate the number of all frames one by one. I find this problem when I try to train a seg-model, and I curious why there are some frames I always get a low accuracy. emmmm, I think may be my problem is in this tfrecord. segment-8956556778987472864_3404_790_3424_790_with_camera_labels.tfrecord It 's the val set of waymo 2d panoptic semantic.

alexzzhu commented 1 year ago

Ah interesting, could you provide the timestamp of the frame(s) that you're seeing this issue?

swimmerQAQ commented 1 year ago

Thanks for your reply. For example 1681388624(1) 19 is my number tfrecord file above. segment-8956556778987472864_3404_790_3424_790_with_camera_labels.tfrecord And this example may be the frame2. I extract it from tfrecord like this format: 8956556778 is 10number of the tfrecord 1681389152(1) my frame number definition ： 1681389298(1)

May some friends can simply check whether is my tfrecord file downloaded with error. Most of other frames are correct.(I just randomly select several and the sky is correct)

alexzzhu commented 1 year ago

Thanks, this may be a data error. I'm traveling at the moment but will look into it asap.

swimmerQAQ commented 1 year ago

Oh, sorry for disturbing your trip. I am glad that it could be of help. I am grateful for your concentration on this issue.

xcyan commented 1 year ago

@swimmerQAQ

19 is my number tfrecord file above. segment-8956556778987472864_3404_790_3424_790_with_camera_labels.tfrecord And this example may be the frame2. Can you confirm the timestamp? I can take a look.

swimmerQAQ commented 1 year ago

from the tf dataset read a frame and get a frame number : 1681953258(1) 1681953422(1)

Does timestamp means frame number? I seemly didn't extract such information. In this tfrecord file, most of seg-labels seem to regard sky as car. Sir, may be you can check several frames randomly.

1681953765(1)

xcyan commented 1 year ago

Thanks for your reply. For example 1681388624(1) 19 is my number tfrecord file above. segment-8956556778987472864_3404_790_3424_790_with_camera_labels.tfrecord And this example may be the frame2. I extract it from tfrecord like this format: 8956556778 is 10number of the tfrecord 1681389152(1) my frame number definition ： 1681389298(1)

May some friends can simply check whether is my tfrecord file downloaded with error. Most of other frames are correct.(I just randomly select several and the sky is correct)

I can confirm the labels for the 2nd frame (pose_timestamp = 1513450821.6389737) are correct.

I assume the problem is either frame decoding function or visualization function on your end, please debug if possible. For our reference decoding and visualization functions, please check the 2d pvps tutorial for details.

More specifically, please check the code piece under Read a single panoptic label.