Can not run the inference using HF

[JessicaLopezEspejel]

Hello, I am having the following error when I run the inference:

ModuleNotFoundError: No module named 'transformers.generation'

Here is the code:

import torch
import transformers
from accelerate import init_empty_weights
from transformers import AutoModelForCausalLM, AutoTokenizer, file_utils,AutoConfig
from accelerate import init_empty_weights, load_checkpoint_and_dispatch, infer_auto_device_map

access_token = "hf_X"

config = AutoConfig.from_pretrained(model_path, trust_remote_code=True, use_auth_token=access_token)
config.init_device = "cuda:0"

model = AutoModelForCausalLM.from_pretrained("bigcode/octogeex", config=config, trust_remote_code=True)

Additionally, here is another error:

Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.

Can you help me to figure out the solution?

Thank you in advance.

[JessicaLopezEspejel]

Hello,

Sorry, the error came from the transformers library version.

Here the installation worked for me:

!pip install typer<0.5.0
!pip install pydantic!=1.8
!pip install protobuf transformers==4.30.2 cpm_kernels torch>=2.0 gradio mdtex2html sentencepiece accelerate