search

ruili3 / Know-Your-Neighbors

[CVPR 2024] 🏡Know Your Neighbors: Improving Single-View Reconstruction via Spatial Vision-Language Reasoning
https://ruili3.github.io/kyn
69 stars 8 forks source link

About the label of the scenes #2

Closed River-Zhang closed 6 months ago

River-Zhang commented 6 months ago

Hi authors! Thanks for your great work! I saw the outdoor_labels.txt in your repository and there are about 20 categories. I wonder how you get these labels. Did you use any models to process the dataset and predict the label or just write the label according to common sense? If I want to use it for indoor scenes, how can I set the labels? Thanks very much!

ruili3 commented 6 months ago

Hi Zechuan,

Thanks for your interest in our work! We collect category names from the Cityscape semantic labels focusing on street views.

We do not pre-process the dataset using other models but directly predict the labels using Lseg by computing cosine similarity between image and language embeddings.

I am not that familiar with indoor scene datasets, but referring to indoor segmentation datasets to collect scene-specific label names is always a viable option, eg, NYU, ScanNet, etc. You can also extract labels from a wider range of categories from ADE20K.

Hope this can help you :)

River-Zhang commented 6 months ago

Thanks for your reply! I'll try it!