search

mesnico / RelationNetworks-CLEVR

A pytorch implementation for "A simple neural network module for relational reasoning", working on the CLEVR dataset
MIT License
87 stars 26 forks source link

Do you know how to calculate the accuracy of the Count, Exists, Compare Numbers, etc #6

Closed ToTheBeginning closed 5 years ago

ToTheBeginning commented 5 years ago

To the best of my knowledge, most repos on github only report the overall accuracy, but I want the same format as the paper reports.

Can you help me, it will be very helpful for me!

mesnico commented 5 years ago

Hi @ToTheBeginning You can find the answer here

Basically, this info is embedded in JSON files encoding questions in the CLEVR dataset. For each question, a functional program formally encodes the question. The last element of the program determines the type of the question.

It should not be difficult to classify questions on the basis of that piece of information. Hope it helps!

ToTheBeginning commented 5 years ago

Thanks. That is very helpful!