Question about communication of Where2Comm

[HuangZhe885]

Thank you very much for your efforts. I'd want to ask a question about communication efficiency of Where2Comm. In my opinion, does not need to transmit feature maps. Isn't it sufficient to convey its own pose and bounding box directly? When others receive this, they should convert it to their own coordinate system and then integrate it to fill in the gaps. Transmission of plain text can significantly cut communication costs. So why don't convey pose and bounding box directly？

[Little-Podi]

Hi. I think you are talking about the late collaboration manner, which has been implemented in several papers as a baseline to compare. You are right about the communication cost reduction part. However, the ultimate goal of current cooperative perception research is to push the performance-bandwidth trade-off and the late collaboration mode may not be optimal for this. You can refer to this survey paper for detailed discussions about different collaboration modes. Here, let me give you a very quick example. Imagine all the vehicles have incomplete observations of an object, then it will probably not be included in the shared bounding boxes since no one can detect this object by its own observation. Thus, the whole multi-agent system will finally fail to detect such objects. However, if the communication-heavy but more informative feature maps are shared, then the partial observations can be gathered togerther to recover a complete object. This will definitely improve the detection performance. In a nutshell, there is always a performance-bandwidth trade-off and that's why we need researcher's efforts to achieve this objective.

[HuangZhe885]

Hi. I think you are talking about the late collaboration manner, which has been implemented in several papers as a baseline to compare. You are right about the communication cost reduction part. However, the ultimate goal of current cooperative perception research is to push the performance-bandwidth trade-off and the late collaboration mode may not be optimal for this. You can refer to this survey paper for detailed discussions about different collaboration modes. Here, let me give you a very quick example. Imagine all the vehicles have incomplete observations of an object, then it will probably not be included in the shared bounding boxes since no one can detect this object by its own observation. Thus, the whole multi-agent system will finally fail to detect such objects. However, if the communication-heavy but more informative feature maps are shared, then the partial observations can be gathered togerther to recover a complete object. This will definitely improve the detection performance. In a nutshell, there is always a performance-bandwidth trade-off and that's why we need researcher's efforts to achieve this objective.

[HuangZhe885]

Hi. I think you are talking about the late collaboration manner, which has been implemented in several papers as a baseline to compare. You are right about the communication cost reduction part. However, the ultimate goal of current cooperative perception research is to push the performance-bandwidth trade-off and the late collaboration mode may not be optimal for this. You can refer to this survey paper for detailed discussions about different collaboration modes. Here, let me give you a very quick example. Imagine all the vehicles have incomplete observations of an object, then it will probably not be included in the shared bounding boxes since no one can detect this object by its own observation. Thus, the whole multi-agent system will finally fail to detect such objects. However, if the communication-heavy but more informative feature maps are shared, then the partial observations can be gathered togerther to recover a complete object. This will definitely improve the detection performance. In a nutshell, there is always a performance-bandwidth trade-off and that's why we need researcher's efforts to achieve this objective.

Thank you so much for your reply!