huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
134.24k stars 26.84k forks source link

Add Molecular Attention Transformer #18990

Closed shivance closed 1 year ago

shivance commented 2 years ago

Model description

I would like to add the Molecular Attention Transformer MAT model to the Transformers

MAT has been a big leap towards development of a single neural network architecture that performs competitively across a range of molecule property prediction tasks. It unlocked a widespread use of deep learning in the drug discovery industry. Key innovation in MAT is to augment the attention mechanism in Transformer using inter-atomic distances and the molecular graph structure. Experiments show that MAT performs competitively on a diverse set of molecular prediction tasks.

Open source status

Provide useful links for the implementation

Link to Model Repo : Link Link to Paper : Link

shivance commented 2 years ago

@sgugger Hi, would it be a valuable contribution to HuggingFace?

sgugger commented 2 years ago

If you want to dive into this model, yes it would definitely be of interest!

shivance commented 2 years ago

Alright @sgugger , I'll soon put up a PR on this !

Bearnardd commented 1 year ago

Hi @shivance! - Are you still working on this model? Since if not I would be interested in picking it up. Have a nice day!

shivance commented 1 year ago

Sure go ahead @Bearnardd