About annotation quality

[qiaozhijian]

There are two pictures selected from segment-9350921499281634194_2403_251_2423_251_with_camera_labels.

The first picture shows the raw image while the second shows the GT lanes which colours represent their categories.

So, you can see the second picture. There is left-curbside, white-solid, white-dashed, yellow, white-solid, white-dashed, white-solid and right-curbside from left to right.

The problem is that the two leftmost lanes are mixed.

[qiaozhijian]

The second picture is on the first timestamp while the first picture is not. I select the first picture just for visualization.

[RicardLee]

Hi, thanks for pointing out problem. Due to limitation of existing labeling pipeline and labeling tools, some annotation error could be caused. Further improvement in the existing quality of labeling will bring a huge amount of work. In future data sets, we will take this seriously.

[qiaozhijian]

Hi, thanks for pointing out problem. Due to limitation of existing labeling pipeline and labeling tools, some annotation error could be caused. Further improvement in the existing quality of labeling will bring a huge amount of work. In future data sets, we will take this seriously.

I guess so. Maybe it's due to the labeling error of Waymo Dataset. Anyway, thanks for your work!

[qiaozhijian]

Another problem is that there is a wrong 3D GT point whose xy is (0,0) in some annotated lanes. These points also affect evaluation.

[RicardLee]

Hi, thanks, we have found that problem. We will fix it asap.

[RicardLee]

Hi, we filter some noise point in new released dataset.