DerrickXuNu / OpenCOOD

[ICRA 2022] An opensource framework for cooperative detection. Official implementation for OPV2V.
https://mobility-lab.seas.ucla.edu/opv2v/
Other
644 stars 99 forks source link

Why is it necessary to compress the features of ego vehicles when compressing feature maps #120

Closed githigher closed 8 months ago

githigher commented 8 months ago

The following is the relevant code for v2xvit:

        # n, 4 -> n, c
        batch_dict = self.pillar_vfe(batch_dict)
        # n, c -> N, C, H, W
        batch_dict = self.scatter(batch_dict)
        batch_dict = self.backbone(batch_dict)

        spatial_features_2d = batch_dict['spatial_features_2d']

        # downsample feature to reduce memory
        if self.shrink_flag:
            spatial_features_2d = self.shrink_conv(spatial_features_2d)
        # compressor
        if self.compression:
            spatial_features_2d = self.naive_compressor(spatial_features_2d)
        # N, C, H, W -> B, L, C, H, W
        regroup_feature, mask = regroup(spatial_features_2d,
                                        record_len,
                                        self.max_cav)

I noticed that spatial_features_2d is a feature map of N, C, H, and W sizes. N represents the number of ego vehicles and other agents. When spatial_features_2d's 0th dimension is equal to 0, it represents the feature map of the ego vehicle. But when you compress, you input the entire feature map, which includes the ego vehicle. I think the characteristics of the ego vehicle itself should not need to be compressed because its transmission loss is 0. Can you tell me why this is?

DerrickXuNu commented 8 months ago

Nice catch! We shouldn't compress the ego feature. It won't influence the performance a lot, but you can change to the right setting in your own experiment.

githigher commented 8 months ago

Haha, thank you very much for your reply. I will try to change it and conduct the experiment later