search

ludwig-ai / ludwig

Low-code framework for building custom LLMs, neural networks, and other AI models
http://ludwig.ai
Apache License 2.0
10.97k stars 1.18k forks source link

Dependency issue #3980

Open robhheise opened 3 months ago

robhheise commented 3 months ago

Describe the bug When running the example here I am getting an exception of raise ImportError( ImportError: Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: `pip install -i https://pypi.org/simple/ bi To Reproduce Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Warnings and other logs: Loading large language model... Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes, retrying in 1 seconds... Loading large language model... Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes, retrying in 2.1122417637009163 seconds... Loading large language model... Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes, retrying in 5.089839704704617 seconds... Loading large language model... Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes, retrying in 11.00884440709334 seconds... Loading large language model... Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes, retrying in 22.967673403551867 seconds... Loading large language model... Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes, retrying in 46.62241237591277 seconds... Loading large language model... Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes, retrying in 93.39312220005493 seconds... Loading large language model... Traceback (most recent call last): File "/Users/robertheise/Documents/SD/kh-accel/ge-llm.py", line 287, in results = model_ft_v1.train(dataset=birdi_df) File "/Users/robertheise/Documents/SD/kh-accel/venv/lib/python3.10/site-packages/ludwig/api.py", line 634, in train self.model = LudwigModel.create_model(self.config_obj, random_seed=random_seed) File "/Users/robertheise/Documents/SD/kh-accel/venv/lib/python3.10/site-packages/ludwig/api.py", line 2083, in create_model return model_type(config_obj, random_seed=random_seed) File "/Users/robertheise/Documents/SD/kh-accel/venv/lib/python3.10/site-packages/ludwig/models/llm.py", line 100, in init self.model = load_pretrained_from_config(self.config_obj, model_config=self.model_config) File "/Users/robertheise/Documents/SD/kh-accel/venv/lib/python3.10/site-packages/decorator.py", line 232, in fun return caller(func, *(extras + args), *kw) File "/Users/robertheise/Documents/SD/kh-accel/venv/lib/python3.10/site-packages/retry/api.py", line 73, in retry_decorator return __retry_internal(partial(f, args, kwargs), exceptions, tries, delay, max_delay, backoff, jitter, File "/Users/robertheise/Documents/SD/kh-accel/venv/lib/python3.10/site-packages/retry/api.py", line 33, in __retry_internal return f() File "/Users/robertheise/Documents/SD/kh-accel/venv/lib/python3.10/site-packages/ludwig/utils/llm_utils.py", line 76, in load_pretrained_from_config model: PreTrainedModel = AutoModelForCausalLM.from_pretrained(pretrained_model_name_or_path, load_kwargs) File "/Users/robertheise/Documents/SD/kh-accel/venv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 563, in from_pretrained return model_class.from_pretrained( File "/Users/robertheise/Documents/SD/kh-accel/venv/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3049, in from_pretrained hf_quantizer.validate_environment( File "/Users/robertheise/Documents/SD/kh-accel/venv/lib/python3.10/site-packages/transformers/quantizers/quantizer_bnb_4bit.py", line 62, in validate_environment raise ImportError( ImportError: Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes

Expected behavior I was hoping that the example would work.

Screenshots If applicable, add screenshots to help explain your problem.

Environment (please complete the following information): I have tried on a MacBook and jupyter hub

Additional context It would be great if the examples provided on predibase website where actually working: https://predibase.com/blog/ludwig-10k-stars-llm-fine-tuning-hackathon-winners

alexsherstinsky commented 3 months ago

Hi @robhheise -- I am not seeing any error stack traces in the notebook attached. Could you please run it and include one with the error you are seeing so we can take the next steps to troubleshoot? Happy to pair to do it together, if makes sense. Also, while we include only the code that runs at the time of the publication, sometimes new issues come up when the various dependency libraries evolve. If you find such situations, please do file an issue, and we would be happy to take a look (ideally together!). Thank you very much.

alexsherstinsky commented 3 months ago

@robhheise I just ran this notebook in my personal account in Google Colab with the T4 GPU. There were no issues. I stopped the first and second fine-tuning jobs and cleared the memory (you can see those cells), and ran the third (best) fully. Please note that I did not uncomment any cells -- I just ran them as they are written. I have done this previously as well (when the competition results were judged), and there were no issues back then as there were none now. Could you please try to run it again under the same conditions as I described and see if you can replicate my results? Thank you very much!