Closed gjm-anban closed 1 year ago
hello,I run instructcodet5p-16b module in CPU ,this is the code
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer import torch checkpoint = "Salesforce/instructcodet5p-16b" device = "cpu" # for GPU usage or "cpu" for CPU usage tokenizer = AutoTokenizer.from_pretrained(checkpoint) model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint, torch_dtype=torch.float16, low_cpu_mem_usage=True, trust_remote_code=True).to(device) encoding = tokenizer("def print_hello_world():", return_tensors="pt").to(device) encoding['decoder_input_ids'] = encoding['input_ids'].clone() outputs = model.generate(**encoding, max_length=15,pad_token_id=tokenizer.eos_token_id) print(tokenizer.decode(outputs[0], skip_special_tokens=True))
but it comeout with an error
I want to know whether this model can support cpu and how to fix the error
I can run in CPU now ,but it takes huge ram. The ram up to 64.9G(the machine ram total 83.5G)
hello,I run instructcodet5p-16b module in CPU ,this is the code
but it comeout with an error![image](https://github.com/salesforce/CodeT5/assets/73465248/ca7dec3a-5226-4d33-942e-3de77c54a223)
I want to know whether this model can support cpu and how to fix the error