search

MILVLG / openvqa

A lightweight, scalable, and general framework for visual question answering research
Apache License 2.0
318 stars 64 forks source link

Attention Grounding for GQA #67

Closed pkhdipraja closed 4 years ago

pkhdipraja commented 4 years ago

Hello, I am interested in getting the grounding results for GQA but it doesn't seem to be supported at the moment. Is there a plan to support this in the future? or maybe pointer on how to extend the current implementation to support this? (I am particularly interested in getting the result for MCAN model)

Thank you.

MIL-VLG commented 4 years ago

If you mean the attention map visualization in the MCAN paper, we currently do not such plan to add this function into openvqa, as we think this is not a generic functionality for other VQA models. Some other 3rdparty repo like this may be helpful for your requirements.

pkhdipraja commented 4 years ago

Not for the visualization, but the attention weights are useful for evaluating grounding on GQA for attention-based models.

MIL-VLG commented 4 years ago

Got it. The additional annotation files for evaluating the grounding performance are not provided for the test set. Therefore, this result can only be tested online.