facebookresearch / clevr-dataset-gen

A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Other
577 stars 204 forks source link

degenerate questions #21

Open rizar opened 5 years ago

rizar commented 5 years ago

The degeneracy check in generate_questions.py is applied only when there is a "raw" relate node in the template. This effectively means that it is only applied to question families defined in templates/single_and.json. Was this also the case when the original dataset was generated?

On a related note, when I generate new questions, many degenerate questions are not detected because of the object uniqueness check. Is often the case that the output of intersection refers to several objects, and yet all of them have the same value for the queried attributed. Is that by design?