Open Chris33Edwards opened 1 year ago
same question
Hi there, please refer to the instructions here to reproduce the HumanEval results.
Below is a quick example to load the model and do the generation based on a given prompt:
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
import torch
checkpoint = "Salesforce/instructcodet5p-16b"
device = "cuda" # for GPU usage or "cpu" for CPU usage
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint,
torch_dtype=torch.float16,
low_cpu_mem_usage=True,
trust_remote_code=True).to(device)
encoding = tokenizer("def print_hello_world():", return_tensors="pt").to(device)
encoding['decoder_input_ids'] = encoding['input_ids'].clone()
outputs = model.generate(**encoding, max_length=15)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Hi there, please refer to the instructions here to reproduce the HumanEval results.
Below is a quick example to load the model and do the generation based on a given prompt:
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer import torch checkpoint = "Salesforce/instructcodet5p-16b" device = "cuda" # for GPU usage or "cpu" for CPU usage tokenizer = AutoTokenizer.from_pretrained(checkpoint) model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint, torch_dtype=torch.float16, low_cpu_mem_usage=True, trust_remote_code=True).to(device) encoding = tokenizer("def print_hello_world():", return_tensors="pt").to(device) encoding['decoder_input_ids'] = encoding['input_ids'].clone() outputs = model.generate(**encoding, max_length=15) print(tokenizer.decode(outputs[0], skip_special_tokens=True))
I appreciate your share of the text-to-generation section, and I am wondering if there is a model with smaller parameters, such as 4b or 6b?
Because I don't have enough video memory, so I set the value of device to 'cpu' and got this error. I hope you can help me solve this problem, thank you very much! Traceback (most recent call last): File "codegen.py", line 16, in outputs = model.generate(encoding, max_length=15) File "/opt/conda/envs/acv/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context return func(args, kwargs) File "/opt/conda/envs/acv/lib/python3.7/site-packages/transformers/generation/utils.py", line 1269, in generate inputs_tensor, model_kwargs, model_input_name File "/opt/conda/envs/acv/lib/python3.7/site-packages/transformers/generation/utils.py", line 634, in _prepare_encoder_decoder_kwargs_for_generation model_kwargs["encoder_outputs"]: ModelOutput = encoder(encoder_kwargs) File "/opt/conda/envs/acv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(input, kwargs) File "/root/.cache/huggingface/modules/transformers_modules/Salesforce/instructcodet5p-16b/70bb08afa3d6f081b347e67752ca8e031a35ac4a/modeling_codet5p.py", line 572, in forward output_attentions=output_attentions, File "/opt/conda/envs/acv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, *kwargs) File "/root/.cache/huggingface/modules/transformers_modules/Salesforce/instructcodet5p-16b/70bb08afa3d6f081b347e67752ca8e031a35ac4a/modeling_codet5p.py", line 325, in forward hidden_states = self.ln_1(hidden_states) File "/opt/conda/envs/acv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(input, **kwargs) File "/opt/conda/envs/acv/lib/python3.7/site-packages/torch/nn/modules/normalization.py", line 190, in forward input, self.normalized_shape, self.weight, self.bias, self.eps) File "/opt/conda/envs/acv/lib/python3.7/site-packages/torch/nn/functional.py", line 2347, in layer_norm return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled) RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'
Facing similar errors.
Yes, we also have smaller models and you may try smaller models for running on CPU. See below for all the models we've released:
220M
and 770M
: codet5p-220m and codet5p-770m.220M
and 770M
that are further tuned on Python subset: codet5p-220m-py and codet5p-770m-py.2B
, 6B
, 16B
: codet5p-2b, codet5p-6b, and codet5p-16b.16B
: instructcodet5p-16b.Changing torch.float16 to torch.float32 might solve the problem I think
Changing torch.float16 to torch.float32 might solve the problem I think
it worked
Thanks to your wonderful work. However, I wonder how can we use CodeT5+ to generate code based on the natural language description?