[MRG] add gated attention layer, tests, and docs

allenai / deep_qa

A deep NLP library, based on Keras / tf, focused on question answering (but useful for other NLP too)

Apache License 2.0

404 stars 133 forks source link

[MRG] add gated attention layer, tests, and docs #184

Closed nelson-liu closed 7 years ago

nelson-liu commented 7 years ago

Added the gated attention layer described in Dhingra et. al 2016

Built docs: API Summary for layers.attention: https://133-47793051-gh.circle-artifacts.com/0/home/ubuntu/deep_qa/doc/_build/html/layers/attention.html#gatedattention Full API Reference for layers.attention: https://133-47793051-gh.circle-artifacts.com/0/home/ubuntu/deep_qa/doc/_build/html/api_doc/deep_qa.layers.attention.html#module-deep_qa.layers.attention.gated_attention

nelson-liu commented 7 years ago

thanks for the reviews, @DeNeutoy and @matt-gardner . I've addressed the comments, and added tests for all three masked cases. For the compute_mask method, I think the right thing to do is just re-return the mask of the document; thoughts?

matt-gardner commented 7 years ago

Yes, returning the document mask seems like the right thing to do here.