search

huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
135.49k stars 27.12k forks source link

Add `Tensor Parallel` support for ALL models #34789

Open ArthurZucker opened 1 week ago

ArthurZucker commented 1 week ago

Just opening this to add support for all models following #34184

Lets bring support to all model! 🤗

It would be great to add the support for more architectures such as

... and many more

For anyone who wants to contribute just open a PR and link it to this issue, and ping me for a review!! 🤗

farrosalferro commented 1 week ago

If it's okay, I want to take the Gemma and Gemma2 @ArthurZucker

kmehant commented 6 days ago

@ArthurZucker I will add support for Granite. Thanks

VladOS95-cyber commented 4 days ago

Hey @ArthurZucker! I am going to work on Mistral.