Relja / netvlad

NetVLAD: CNN architecture for weakly supervised place recognition
MIT License
534 stars 122 forks source link

Applying to Indoor usage #40

Closed oym1994 closed 3 years ago

oym1994 commented 3 years ago

Hi, thanks for your great contribution to place recognition. Here I want to consult some questions about indoor usage.

In the paper, all experiments are conducted in outdoor datasets. I wander what would be the difference if applying it into a real indoor environment to help re-localization and loop detection in a SLAM system, such as home, official and even some room with repeat region(such as a Server Room where many server computers with the same appearance are placed)? What should I care about when collecting training data without GPS? Thanks for your attention and I am always looking forward to your kind response and any advice.

Best , Slamer

Relja commented 3 years ago

Hi,

Indeed it's not as straight forward to download a dataset of indoor images. But it's not necessary to have GPS coordinates, on a more high level you just need some way of getting potential positives Vs negatives, and this can be done even without GPS, e.g. download all images from restaurants, all images of one restaurant are potential positives and of different restaurants are negatives. We did something similar in Neighbourhood Consensus Networks https://arxiv.org/abs/1810.10510 see appendix E, but we used it to train matching, not NetVLAD - a larger dataset is probably needed for NetVLAD training.

NetVLAD has been applied on indoors datasets later, e.g. see https://arxiv.org/abs/1803.10368 where they combine it with spatial verification. This used our pre trained networks, I'm not sure if anyone actually trained NetVLAD on indoors data. Pretrained seems to work decently enough but I'm sure with enough good indoors training data that it would do even better. But indeed you are right, indoors is quite tough.

Best, Relja

oym1994 commented 3 years ago

Got it! Thanks for your kind response!