a stronger version of Where2comm

[Little-Podi]

This version should have stronger performance. I don't remember the best configutation, but here are some configurations can be tried to see how it goes:

• Set where2comm_fusion.fully to True during training and False during inference (pruning the communication graph by a pre-defined threshold)
• Set where2comm_fusion.fully to False during training (random sampling communication ratios, I haven't implemented the incremental learning as the original paper)
• Set where2comm_fusion.multi_scale to False (previous version has bugs in single-scale fusion, and I have corrected it)

[DerrickXuNu]

Quick question: When we set fully to true, what is the compression rate? Is it something random?

[Little-Podi]

Where2comm reduces the communication cost by selecting spatially sparse yet perceptually critical areas. where2comm_fusion.fully denotes whether we keep the fully-connected communication graph or prune it. In the updated implementation, it works like this:

• During training, if where2comm_fusion.fully is True, then the model will be always trained with 100% communication without any communication saving. Otherwise, the pruning ratio will be randomly selected from 0% to 100%.
• During inference, if where2comm_fusion.fully is True, then the agent will send the whole feature map to the ego agent. Otherwise, it will send partial features according to the confidence map and the pre-defined threshold.

[DerrickXuNu]

Where2comm reduces the communication cost by selecting spatially sparse yet perceptually critical areas. where2comm_fusion.fully denotes whether we keep the fully-connected communication graph or prune it. In the updated implementation, it works like this:

• During training, if where2comm_fusion.fully is True, then the model will be always trained with 100% communication without any communication saving. Otherwise, the pruning ratio will be randomly selected from 0% to 100%.
• During inference, if where2comm_fusion.fully is True, then the agent will send the whole feature map to the ego agent. Otherwise, it will send partial features according to the confidence map and the pre-defined threshold.

Thanks for the clarification. Do you know what kind of threshold should we set to be the same as the v2x-vit paper, e.g., 32x compression rate?

[Little-Podi]

I think 32x will be fine. In Where2comm, they didn't use channel-wise compression to save the communication cost, so I didn't try it.

[DerrickXuNu]

I think 32x will be fine. In Where2comm, they didn't use channel-wise compression to save the communication cost, so I didn't try it.

I may not ask the question clearly. Other models by default use 32x compression, which will impact the performance. To make fair comparison, where2comm should do the same thing. But it seem they already have compression inside the algorithm, so I am not sure how much compression they have by default/

[Little-Podi]

For a pair comparison, you can use 32x. It will make the channel dimension of the transmitted feature map the same as other models. Besides, in my practice, the threshold 0.01 already filter out a large portion of the feature map, which further reduce the communication cost.

[Little-Podi]

Oh, I got your point. Because the "compression" mechanism in Where2comm is based on a threshold, it cannot precisely control the ratio such as 1/32 or something. By default, they use 0.01.

[Little-Podi]

You can tune this threshold after the model is trained to make the compression rate to be roughly 32x. But the exact rate will be slightly different between different inputs and datasets.

[DerrickXuNu]

Thanks for your reply. I think everything looks good to me now except the "kernel" typo. Could you revert back and update it so I can merge yours into my branch?

[Little-Podi]

Sorry, I was preparing a tutorial yesterday. Thanks for your help, anyway.