search

yashbhalgat / Contrastive-Lift

[NeurIPS 2023 Spotlight] Code for "Contrastive Lift: 3D Object Instance Segmentation by Slow-Fast Contrastive Fusion"
https://www.robots.ox.ac.uk/~vgg/research/contrastive-lift/
59 stars 1 forks source link

Poor performance on duplicate objects #7

Open gtangg12 opened 6 months ago

gtangg12 commented 6 months ago

Hello! Really good work.

I was benchmarking your method wrt to Replica (i set the batch size to 4096 and segment loss to 0.75 as they did in Panopli; i also increased max instances to 25 which is also used in Panopli).

It seems the PQ is similar (perhaps due to semantics performing well?), but CL often is able to delineate duplicate adjacent objects. For example, here is result from training on replica-vmap office3 split (slow-fast, above) compared to Panopli (linear assignment; below). Do you have any intuition why this is the case? Thank you!

CL: image image

PL: image image

gtangg12 commented 5 months ago

Were the Replica scenes run with max_instances = 3, as stated in the config? I was only able to replicate the results with max_instances = 3 but not 25, which is used in Panoptic Lifting. However, running it with 3 doesn't make sense since there are many times more than 3 objects in the scene? Thanks!