search

guillaume-be / rust-bert

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)
https://docs.rs/crate/rust-bert
Apache License 2.0
2.64k stars 215 forks source link

Bloom model integration #276

Open Miezhiko opened 2 years ago

Miezhiko commented 2 years ago

maybe

there it is https://huggingface.co/bigscience/bloom

guillaume-be commented 2 years ago

Hello @Miezhiko ,

Thank you for the suggestion - I will look into it in the next few weeks. I will probably target the smaller models for an initial implementation as I don't have access to a machine which can load the full size model at the moment.

danforbes commented 1 year ago

What is the status of this effort?

guillaume-be commented 1 year ago

Hello @danforbes , I haven't had the chance to look into this yet, but would have some capacity starting next week. Is that something you would like to look into?

danforbes commented 1 year ago

Honestly, I would love to, but I'm afraid I would need a lot of help and may not be the best person for the job - I'm not a terribly experienced Rust developer, nor do I really understand all that much about LLMs 😅 At this point I would mostly describe myself as an enthusiast that happens to have a fair amount of programming experience 🤓 I recently authored this PR on the rustformers/llama-rs repository, which was a bit of a basic refactor and a lot of copy-pasta 😅 It did get me wondering if there are opportunities for that project to have some type of interaction with this one, though 🤔 That PR also has a fair amount of code related to BLOOM inference, so it may be helpful in integrating BLOOM with this project. Let me know if you have any thoughts on the above.