search

ELS-RD / transformer-deploy

Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀
https://els-rd.github.io/transformer-deploy/
Apache License 2.0
1.64k stars 150 forks source link

GPTModelWrapper' object has no attribute 'can_generate' #177

Open jxcomeon opened 1 year ago

jxcomeon commented 1 year ago

When i run 

with torch.inference_mode():
    gpt2_model = GPTModelWrapper(config=model.config, device=model.device, inference=inference_torch)
    sample_output = gpt2_model.generate(inputs.input_ids, max_length=64)

There has an error 'GPTModelWrapper' object has no attribute 'can_generate' how can i do? thanks

ostlee commented 12 months ago

When i run

with torch.inference_mode():
    gpt2_model = GPTModelWrapper(config=model.config, device=model.device, inference=inference_torch)
    sample_output = gpt2_model.generate(inputs.input_ids, max_length=64)

There has an error 'GPTModelWrapper' object has no attribute 'can_generate' how can i do? thanks

I encountered the same problem. The reason is that the GPTModelWrapper, which inherits from the GenerationMixin, lacks the corresponding class methods and related attributes. My solution is to modify the GPTModelWrapper class according to the PreTrainedModel in transformers : 1. Add the _cangenerate() method 2. Add self.generation_config in init()