Open BEILOP opened 1 year ago
You should try to update transformers (>= 4.31.0.dev0) , accelerate (>=0.21.0.dev0) and bitsandbytes. And instead of loading in 8 bit, try to use
model = AutoModelForCausalLM.from_pretrained("bigcode/starcoder", device_map="auto", load_in_4bit=True)
how to change this code to make it work on 2 11G 2080ti?: