ablation study reproduction

[billbliss3]

Hi author:

StreamMapNet is a awesome work! Recently, I am confused by the ablation study in your paper. I use nus R50 new split with both Query propagation and BEV fusion. https://github.com/yuantianyuan01/StreamMapNet/blob/main/plugin/configs/nusc_newsplit_480_60x30_24e.py Then I turn off them one by one.

• nus R50 new split, Query propagation and BEV fusion mAP 0.3387

• nus R50 new split, BEV fusion mAP 0.3389

• nus R50 new split, Query propagation mAP 0.3138

• nus R50 new split mAP 0.3096

It seems that the Query propagation is nearly useless.

Could you share some config of ablation study for us. Thanks.

[TonyPhh]

hi， my results are similar to yours

nus R50 new split, Query propagation and BEV fusion mAP 0.3376

nus R50 new split, BEV fusion mAP 0.3414

[yuantianyuan01]

It seems quite interesting that BEV fusion may saturate the effect of query propagation when evaluated on NuScenes. In fact, our ablation studies were conducted on Argoverse2 with 100x50m range. Could you try experiments on this setting?