search

yuantianyuan01 / StreamMapNet

GNU General Public License v3.0
186 stars 16 forks source link

Performance degradation based on maptr #29

Open charmerfuerl opened 1 month ago

charmerfuerl commented 1 month ago

Thanks for your great work! In terms of the single frame prediction, i think the code difference between maptr and streammapnet should be the remove of point queries and the multipoint attention, is there any other difference between them? Are the pre-process and post-process the same with maptr? I directly changed the code based on the maptrv2 code but found a huge performance degradation(almost 20% degradation).

yuantianyuan01 commented 1 month ago

Hi, thanks for your interest in our project.

We use a different BEV coordinate system, which may be the root of the performance gap you encountered. Specifically, please check the coordinate reversion in bev encoder and the one in attention layer.

charmerfuerl commented 1 month ago

Hi, thanks for your interest in our project.

We use a different BEV coordinate system, which may be the root of the performance gap you encountered. Specifically, please check the coordinate reversion in bev encoder and the one in attention layer.

Hi, Thank you for your reply, i tried not to change the bev encoder, but also to cancel the coordinate reversion in attention layer, but still got a huge performance gap, is the reversion in bev encoder and attention layer necessary?

yuantianyuan01 commented 1 month ago

Other places may also be affected by the reversed y-axis, for example the temporal BEV fusion: https://github.com/yuantianyuan01/StreamMapNet/blob/main/plugin/models/mapers/StreamMapNet.py#L66.

charmerfuerl commented 1 month ago

Thank you so much! i have got the correct map. But when i changed the head embed_dims from 512 to 256, just like the maptr, the MAP decreased by 6%, so can i think it as one of the key points of the streammap method?

yuantianyuan01 commented 1 month ago

That's very interesting and beyond my expectation. Which config file did you use btw?

charmerfuerl commented 1 month ago

That's very interesting and beyond my expectation. Which config file did you use btw? I realized the single-frame streammap based on maptrv2, https://github.com/hustvl/MapTR. Based on the NuScenes oldsplit, the map was up to 61%, but when i changed the embed_dims from 512 to 256, without any other changes, the map was 54%.

charmerfuerl commented 1 month ago

That's very interesting and beyond my expectation. Which config file did you use btw?

I realized the single-frame streammap based on maptrv2, https://github.com/hustvl/MapTR. Based on the NuScenes oldsplit, the map was up to 61%, but when i changed the embed_dims from 512 to 256, without any other changes, the map was 54%.

Cxb0219 commented 1 month ago

That's very interesting and beyond my expectation. Which config file did you use btw?

I realized the single-frame streammap based on maptrv2, https://github.com/hustvl/MapTR. Based on the NuScenes oldsplit, the map was up to 61%, but when i changed the embed_dims from 512 to 256, without any other changes, the map was 54%.

Hello, I am also trying to implement Maptr-based streammmap recently, but I encountered some problems in the configuration process. Could you please give me some guidance?

charmerfuerl commented 1 month ago

That's very interesting and beyond my expectation. Which config file did you use btw?

I realized the single-frame streammap based on maptrv2, https://github.com/hustvl/MapTR. Based on the NuScenes oldsplit, the map was up to 61%, but when i changed the embed_dims from 512 to 256, without any other changes, the map was 54%.

Hello, I am also trying to implement Maptr-based streammmap recently, but I encountered some problems in the configuration process. Could you please give me some guidance?

Of course, how can i help you?