Add EMAP Demo Notebook with LXMERT

pliang279 / MultiViz

[ICLR 2023] MultiViz: Towards Visualizing and Understanding Multimodal Models

MIT License

92 stars 5 forks source link

Add EMAP Demo Notebook with LXMERT #10

Closed gchhablani closed 3 years ago

gchhablani commented 3 years ago

This PR adds EMAP Demo notebook and closes #7.

The colab notebook can be viewed here.

gchhablani commented 3 years ago

There is an issue with the current implementation of EMAP mentioned in #14. Until this issue is fixed, the demo notebook cannot be made properly.

gchhablani commented 3 years ago

The issue #14 is fixed.

The demo notebook had been added. For the subset chosen, atleast, the EMAP scores are equal to original scores which means that the model is not utilizing cross-modal interactions as suggested by the paper.