search

facebookresearch / ContrastiveSceneContexts

Code for CVPR 2021 oral paper "Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts"
MIT License
218 stars 29 forks source link

Intuition for Contrastive Scene Contexts #9

Closed Divadi closed 3 years ago

Divadi commented 3 years ago

Hi! Thank you for your amazing work.

I wanted check if I understood why the Contrastive Scene Contexts approach works much better than PointContrast, especially for more sampled points. Would it be correct to think that using lots of points for PointContrast does not have significant benefits because many of those points are "easy negatives?" Then, separating the scene into partitions and equally weighting each partition more explicitly forces the model to also learn to differentiate against negatives of different types (close/far, relative angles), some of which are harder negatives. Is this understanding reasonable? Thank you for your time

Sekunde commented 3 years ago

Hi! Thanks for your interest in our work.

I think that can be interpreted differently and yours sounds reasonable to me.

We want to embed the spatial information in the training. But you can also understand it as hard negative mining with the help of spatial contexts. In comparison, hard contrastive loss (defined in PointContrast) finds the hard negative pairs that are closest to the positive in the feature space, however, it is unstable and hard to train. And the spatial partitions have the balancing effect when more sampled points are available, leading to more hard/types of negatives as you mentioned, e.g. in closer regions (indeed the points are closed spatially have more similar features).

Divadi commented 3 years ago

I understand, thank you so much!