Add Molecular Attention Transformer

[shivance]

Model description

I would like to add the Molecular Attention Transformer MAT model to the Transformers

MAT has been a big leap towards development of a single neural network architecture that performs competitively across a range of molecule property prediction tasks. It unlocked a widespread use of deep learning in the drug discovery industry. Key innovation in MAT is to augment the attention mechanism in Transformer using inter-atomic distances and the molecular graph structure. Experiments show that MAT performs competitively on a diverse set of molecular prediction tasks.

Open source status

• [X] The model implementation is available
• [X] The model weights are available

Provide useful links for the implementation

Link to Model Repo : Link Link to Paper : Link

[shivance]

@sgugger Hi, would it be a valuable contribution to HuggingFace?

[sgugger]

If you want to dive into this model, yes it would definitely be of interest!

[shivance]

Alright @sgugger , I'll soon put up a PR on this !

[Bearnardd]

Hi @shivance! - Are you still working on this model? Since if not I would be interested in picking it up. Have a nice day!

[shivance]

Sure go ahead @Bearnardd