Clarification on Comparative Method Performance

[fjhzhixi]

Thank you for your excellent work. While reading your article, I have some queries regarding the performance of other methods as presented.

• I noticed some inconsistencies in the performance testing of the where2comm method on DAIR-V2X as shown in your article's Figure 4 (1) compared to the supplementary material of the original where2comm paper.

  

• there are discrepancies between the accuracy of comparative methods in Table 1 of your paper and Table 1 in your previous article titled "Spatio-Temporal Domain Awareness for Multi-Agent Collaborative Perception."

  

Could you please provide some insight into the standards or criteria used to obtain the performance metrics of these comparative methods? Your clarification on this matter would be greatly appreciated.

[ydk122024]

Thank you for your attention! I answer your queries below:

• Inconsistencies in the results of the where2comm on the DAIR-V2X dataset.

The inconsistencies potentially stem from two aspects: (1) we report the reproduced results for where2comm; (2) compared to their original reports, our performance testing is performed under the noisy setting that includes the transmission delay.

• Discrepancies between the comparison results in Table 1 from the NeurIPS and ICCV papers.

The discrepancies come from the different settings of the two papers. Compared to ICCV, transmission delay is considered in NeurIPS. Also, all experiments in NeurIPS are constrained to ≈ 1 MB bandwidth consumption to reflect the narrow communication channels in real V2V/X scenarios.

[fjhzhixi]

Thanks for your answer, just one more question, what strategy do you use to compress the bandwidth of methods other than where2comm? As far as I know, their method does not support dynamic adjustment of bandwidth.

[ydk122024]

Insightful finding. In practice, we chose a suitable compression rate to make the compressed features close to 1MB across other methods.