search

hustvl / MapTR

[ICLR'23 Spotlight] MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction
MIT License
984 stars 152 forks source link

About Maptr_v1 centerline #90

Open TonyXuQAQ opened 10 months ago

TonyXuQAQ commented 10 months ago

It's impressive that MapTR-V2 improves 3 road element detection scores by 10%. However it seems that there are no results reported about centerline detection by Maptr_v1. May I know the comparison between MapTR-V1 and MapTR-V2 on the centerline element?

LegendBC commented 10 months ago

Sorry for that, we have not conducted this experiment. Since how to support additional fixed-order centerline beyond permutation-equivalent semantics (ped, divider, boundary) with unified modeling and matching is the contribution of MapTRv2.

And we are glad to conduct this comparison on the architecture level in the coming days. We will inform you of the corresponding results in this thread.

zyc10ud commented 9 months ago

@TonyXuQAQ Now we have trained MapTRv1 with extra semantic centerline on Nuscenes dataset, and the metrics is as follows ----------threshhold:0.5---------- cls:divider done in 7.432631s!! cls:ped_crossing done in 1.256378s!! cls:boundary done in 3.170735s!! cls:centerline done in 8.767859s!!

+--------------+-------+--------+--------+-------+ | class | gts | dets | recall | ap | +--------------+-------+--------+--------+-------+ | divider | 27332 | 93902 | 0.431 | 0.225 | | ped_crossing | 6406 | 24888 | 0.214 | 0.125 | | boundary | 21050 | 66516 | 0.345 | 0.201 | | centerline | 48536 | 115644 | 0.377 | 0.203 | +--------------+-------+--------+--------+-------+ | mAP | | | | 0.188 | +--------------+-------+--------+--------+-------+ ----------threshhold:1.0---------- cls:divider done in 7.445451s!! cls:ped_crossing done in 1.248837s!! cls:boundary done in 3.158861s!! cls:centerline done in 9.104348s!!

+--------------+-------+--------+--------+-------+ | class | gts | dets | recall | ap | +--------------+-------+--------+--------+-------+ | divider | 27332 | 93902 | 0.658 | 0.440 | | ped_crossing | 6406 | 24888 | 0.478 | 0.343 | | boundary | 21050 | 66516 | 0.664 | 0.529 | | centerline | 48536 | 115644 | 0.618 | 0.454 | +--------------+-------+--------+--------+-------+ | mAP | | | | 0.441 | +--------------+-------+--------+--------+-------+ ----------threshhold:1.5---------- cls:divider done in 7.441878s!! cls:ped_crossing done in 1.226514s!! cls:boundary done in 3.064285s!! cls:centerline done in 8.831834s!!

+--------------+-------+--------+--------+-------+ | class | gts | dets | recall | ap | +--------------+-------+--------+--------+-------+ | divider | 27332 | 93902 | 0.777 | 0.583 | | ped_crossing | 6406 | 24888 | 0.674 | 0.545 | | boundary | 21050 | 66516 | 0.799 | 0.699 | | centerline | 48536 | 115644 | 0.729 | 0.591 | +--------------+-------+--------+--------+-------+ | mAP | | | | 0.604 | +--------------+-------+--------+--------+-------+ divider: 0.41592279076576233 ped_crossing: 0.33753739794095355 boundary: 0.476189245780309 centerline: 0.4160437136888504 map: 0.4114232870439688

hanbrianlee commented 5 months ago

Hi. As far as I understand, supporting additional fixed-order centerline in MapTRv2 is really just defining one unique permutation as opposed to multiple permutations done for other types (based on detected class).

When you trained the centerline with MapTRv1, did you train it with 2 possible permutations for the centerline? (basically treating centerline as polyline like lane dividers or boundary)?