autonomousvision / giraffe

This repository contains the code for the CVPR 2021 paper "GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields"
https://m-niemeyer.github.io/project-pages/giraffe/index.html
MIT License
1.23k stars 160 forks source link

About multi-object disentanglement and how to set the list probs #33

Closed Ganlin-Yang closed 3 years ago

Ganlin-Yang commented 3 years ago

Hi, I'm very interested in your extraordinary work! And now I'm trying to verify how the model can handle the multi-object cases. Now I'm using the dataset Clevr2345 to train from scratch, but I set the n_boxes to 4 in config file to see what the network can learn when the branches of n_boxes set in the model is smaller than that in pictures from dataset. But I came into a problem in im2scene/giraffe/models/generator.py, there is a function named get_object_existance. In your code, you hard-code the list probs to only handle n_boxes=5 cases, I want to know how I should change the list probs for my purpose, and what is the mathematical meaning of this list. Wish to get your reply, thanks.

m-niemeyer commented 3 years ago

Hi @Ganlin-Yang , yes, you are correct. Please see the comment we added as well to describe this. How you need to change this code depends on what you want to do exactly. You can see here that the function is only called if sample_object_existance is set to true. If you always want to use e.g. 4 boxes, you could disable this. If you want to manipulate the probabilities themselves, you need to change the hard-coded probability list. The current mathematical meaning is that each item in the list defines the probability for one event, and in our Clevr2345 case, the events are "2 boxes", "3 boxes", "4 boxes", "5 boxes". You can see here that the lowest number we allow are 2 boxes.