ValueError: cannot reshape array of size <number> into shape

[TheSlackOne]

System Info

Python 3.11.2 Debian 12 ctransformers 0.2.27 transformers 4.42.4

Who can help?

@ArthurZucker

Information

• [X] The official example scripts
• [ ] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

Following the example on Hugging Face: https://huggingface.co/docs/transformers/en/gguf

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from huggingface_hub import hf_hub_download
from llama_cpp import Llama

# Ensure the model is loaded on the GPU
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print(f"device set to: {device}")

# Load the model and tokenizer
model_name = "TheBloke/dolphin-2.0-mistral-7B-GGUF"
model_file = "dolphin-2.0-mistral-7b.Q4_K_M.gguf"
print(f"Loading tokenizer from {model_name}.")
tokenizer = AutoTokenizer.from_pretrained(model_name, gguf_file=model_file)
print(f"Loading model from repo {model_name}. Model file name {model_file}.")
model = AutoModelForCausalLM.from_pretrained(model_name, gguf_file=model_file)

I get

ValueError: cannot reshape array of size 36864000 into shape (222,72)

Expected behavior

Model should be loaded.

[nimadez]

Problem solved for me, see this commit and apply the changes or wait for the next update.

[ArthurZucker]

I think it was part of 4.43! 🤗

[Tangent-90C]

Why do I still have this bug when I use the latest version?

• Python 3.10.14
• Windows 11
• ctransformers 0.2.27
• transformers 4.43.4
• gguf 0.9.1

[ArthurZucker]

can you print import transformers;transformers.__version__ ? 🤗

[nimadez]

I can confirm that it works on 4.44.0

[PrasannaPrabhakar666]

I faced the same problem. Initially I was using Transformers 4.41 ig and upgraded to 4.44. Still faced the same problem. Looked at the code and had same commits. So, cleared cache and restarted the system, it worked.

[ArthurZucker]

Cool, closing as solved!

[TheSlackOne]

Cool, closing as solved!

Hey @ArthurZucker, please, could you explain how it is solved?

[ArthurZucker]

cleared cache and restarted the system, it worked. 😉