A ImportError when I run the program "FinGPT_Training_LoRA_with_ChatGLM2_6B_for_Beginners.ipynb"

[YRookieBoy]

Hi， When I try to run "FinGPT_Training_LoRA_with_ChatGLM2_6B_for_Beginners.ipynb" in google colab, I came aross a problem. The code is

model_name = "THUDM/chatglm2-6b" tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True) model = AutoModel.from_pretrained( model_name, quantization_config=q_config, trust_remote_code=True, device='cuda' )

and, the error is

ImportError: Using load_in_8bit=True requires Accelerate: pip install accelerate and the latest version of bitsandbytes pip install -i https://test.pypi.org/simple/ bitsandbytes or pip install bitsandbytes` model = prepare_model_for_int8_training(model, use_gradient_checkpointing=True)

Last, I program the code in Gcolab pro and I am sure both packages is installed. Please help me solve the problem, thank you so much!

[llk010502]

Hi, based on my experience, you can try to reinstall these two packages when this error shows, then restart your kernel to run your code. Hope this works.

[YRookieBoy]

Thank you very much! I have already run the code successfully.

[Siddharth-Latthe-07]

The error indicates that the necessary packages for 8-bit training, specifically accelerate and bitsandbytes, are either not installed correctly or not recognized by the environment. Here's how you can troubleshoot and resolve the issue:

1. Ensure correct installation
2. Restart the runtime: Sometimes, after installing new packages, you need to restart the runtime for the changes to take effect.
3. Check for correct version and Load the packages before model definition sample snippet

   
   # Install the necessary packages
   !pip install accelerate
   !pip install -i https://test.pypi.org/simple/ bitsandbytes

Restart runtime after installing the packages (manual step in the Colab interface)

Import the required libraries

from transformers import AutoTokenizer, AutoModel from accelerate import Accelerator

Ensure the runtime is using GPU

import torch device = 'cuda' if torch.cuda.is_available() else 'cpu'

Load the model with the necessary configuration

model_name = "THUDM/chatglm2-6b" tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True) model = AutoModel.from_pretrained( model_name, quantization_config=q_config, trust_remote_code=True, device=device )

Prepare model for 8-bit training

from transformers import prepare_model_for_int8_training model = prepare_model_for_int8_training(model, use_gradient_checkpointing=True)

and also check the gpu settings

hope this will help, let me know the further updates
Thanks