search

NouamaneTazi / bloomz.cpp

C++ implementation for BLOOM
MIT License
812 stars 65 forks source link

There was a problem loading the model #16

Open ScottishFold007 opened 1 year ago

ScottishFold007 commented 1 year ago

after I do "python3 convert-hf-to-ggml.py bigscience/bloomz-7b1 ./models ",there was a problem loading the model:

model_path = "/aidata/yh/BelleGroup_BELLE-7B-1M-fp16/" # You can modify the path for storing the local model
    model =  AutoModelForCausalLM.from_pretrained(model_path,from_tf=True)
    model = model.half().cuda()
    tokenizer = AutoTokenizer.from_pretrained(model_path)
    return model,tokenizer

Traceback (most recent call last): File "/root/anaconda3/lib/python3.9/site-packages/streamlit/scriptrunner/script_runner.py", line 557, in _run_script exec(code, module.dict) File "lianjie_web.py", line 20, in model,tokenizer= load_model() File "/root/anaconda3/lib/python3.9/site-packages/streamlit/legacy_caching/caching.py", line 573, in wrapped_func return get_or_create_cached_value() File "/root/anaconda3/lib/python3.9/site-packages/streamlit/legacy_caching/caching.py", line 557, in get_or_create_cached_value return_value = func(*args, **kwargs) File "lianjie_web.py", line 15, in load_model model = AutoModelForCausalLM.from_pretrained(model_path,from_tf=True) File "/root/anaconda3/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 471, in from_pretrained return model_class.from_pretrained( File "/root/anaconda3/lib/python3.9/site-packages/transformers/modeling_utils.py", line 2612, in from_pretrained model, loading_info = load_tf2_checkpoint_in_pytorch_model( File "/root/anaconda3/lib/python3.9/site-packages/transformers/modeling_tf_pytorch_utils.py", line 401, in load_tf2_checkpoint_in_pytorch_model from .modeling_tf_utils import load_tf_weights File "/root/anaconda3/lib/python3.9/site-packages/transformers/modeling_tf_utils.py", line 40, in from .generation import GenerationConfig, TFGenerationMixin ImportError: cannot import name 'TFGenerationMixin' from 'transformers.generation' (/root/anaconda3/lib/python3.9/site-packages/transformers/generation/init.py)

What is the reason for this, please?

NouamaneTazi commented 1 year ago

It seems you're using from_tf=True I found this issue that is related to your case: https://github.com/huggingface/transformers/issues/20611 Can you try the fix in there please?