No implement of generate() when using models from hugging face.

BlackSamorez / tensor_parallel

Automatically split your PyTorch models on multiple GPUs for training & inference

MIT License

619 stars 38 forks source link

No implement of generate() when using models from hugging face. #130

Open 342215448 opened 10 months ago

342215448 commented 10 months ago

What an amazing work, however, when I tried to inference kosmos model from hugging face, there was an error: NotImplementedError: A model class needs to define a prepare_inputs_for_generation method in order to use .generate().