DerrickXuNu / OpenCOOD

[ICRA 2022] An opensource framework for cooperative detection. Official implementation for OPV2V.
https://mobility-lab.seas.ucla.edu/opv2v/
Other
674 stars 100 forks source link

Why is it necessary to compress the features of ego vehicles when compressing feature maps #120

Closed githigher closed 11 months ago

githigher commented 11 months ago

The following is the relevant code for v2xvit:

        # n, 4 -> n, c
        batch_dict = self.pillar_vfe(batch_dict)
        # n, c -> N, C, H, W
        batch_dict = self.scatter(batch_dict)
        batch_dict = self.backbone(batch_dict)

        spatial_features_2d = batch_dict['spatial_features_2d']

        # downsample feature to reduce memory
        if self.shrink_flag:
            spatial_features_2d = self.shrink_conv(spatial_features_2d)
        # compressor
        if self.compression:
            spatial_features_2d = self.naive_compressor(spatial_features_2d)
        # N, C, H, W -> B, L, C, H, W
        regroup_feature, mask = regroup(spatial_features_2d,
                                        record_len,
                                        self.max_cav)

I noticed that spatial_features_2d is a feature map of N, C, H, and W sizes. N represents the number of ego vehicles and other agents. When spatial_features_2d's 0th dimension is equal to 0, it represents the feature map of the ego vehicle. But when you compress, you input the entire feature map, which includes the ego vehicle. I think the characteristics of the ego vehicle itself should not need to be compressed because its transmission loss is 0. Can you tell me why this is?

DerrickXuNu commented 11 months ago

Nice catch! We shouldn't compress the ego feature. It won't influence the performance a lot, but you can change to the right setting in your own experiment.

githigher commented 11 months ago

Haha, thank you very much for your reply. I will try to change it and conduct the experiment later