search

mosaicml / llm-foundry

LLM training code for Databricks foundation models
https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm
Apache License 2.0
4.06k stars 531 forks source link

mosaicml-turbo: Where to find the repo? #565

Open agarvic opened 1 year ago

agarvic commented 1 year ago

I'm trying to implement DecoupledLionW_8bit in my fine-tuning script, but I get the following error:

ERROR: Could not find a version that satisfies the requirement mosaicml-turbo<0.1,>=0.0.2; extra == "gpu" (from llm-foundry[gpu]) (from versions: none)

I haven't been able to find this repo anywhere. Can you please, help me?

dblalock commented 1 year ago

The repo is currently private since we're not sure where our custom kernels should live yet. What environment are you running in (OS, presence of CUDA gpus, python version, etc)? The wheel exists on pypi, so it's a question of whether your setup supports it.

agarvic commented 1 year ago

Thanks for your response. I have finally managed to install it from the wheel after a few changes in my setup to match the requirements :-)

dblalock commented 1 year ago

awesome! Any insights on what needed to change? Would love to make this seamless for everyone else in the future.

tedfeng424 commented 1 year ago

Hello, I ran into a similar problem, I am wondering what changes you made to install this package. Thanks

surak commented 10 months ago

The repo is currently private since we're not sure where our custom kernels should live yet. What environment are you running in (OS, presence of CUDA gpus, python version, etc)? The wheel exists on pypi, so it's a question of whether your setup supports it.

In our supercomputing center we use python 3.11 and 3.12. Currently with around 5000 A100 GPUs using CUDA 12, with some 15000 more to come - probably with cuda 13, as they'll be GH200. So the cp310 wheel won't work for us. Would it be possible to have the source back on your repo as you had before? Thanks

janEbert commented 10 months ago

What's the future direction regarding mosaicml-turbo and llm-foundry? I get a bad feeling about investing in this code base if parts of it (in this case dependencies) are not open source. I know at the moment, mosaicml-turbo is only used for the Lion optimizer, but this could change in the future. I'd really appreciate hearing your (@dblalock and the rest of MosaicML) current thoughts about this, since I'm currently evaluating llm-foundry for some large-scale runs and have had a blast with its simplicity and performance.

jfrankle commented 10 months ago

Hi @janEbert and company - Oops - you caught us! We've been working on a package of kernels called mosaicml-turbo, and we inadvertently released it too soon. It's still a work-in-progress, and we've been trying to keep it quiet until we're ready for an official release. We'll be maintaining and building on llm-foundry (and releasing new tools like mosaicml-turbo) for a very long time to come. We just jumped the gun on this one 😬

Jonathan Frankle

janEbert commented 10 months ago

Thank you Jonathan for the timely response! I'll be sure to keep quiet as well. ;) Great to hear that you're focusing on making llm-foundry even faster; I'm looking forward to what other improvements you manage to achieve in this already performant code base.

Have you decided whether mosaicml-turbo will be open-sourced upon release?