Open apepkuss opened 3 weeks ago
I confirm the problem. There are two reasons why the tokenized chktxt
hash doesn't match the llama-3 one:
chktxt
is also different.@apepkuss If you want, you can try this quick fix:
diff --git a/convert_hf_to_gguf.py b/convert_hf_to_gguf.py
index 6a1a3a93..f1f145b6 100755
--- a/convert_hf_to_gguf.py
+++ b/convert_hf_to_gguf.py
@@ -599,6 +599,8 @@ class Model:
if chkhsh == "4e2b24cc4770243d65a2c9ec19770a72f08cffc161adbb73fcbb6b7dd45a0aae":
# ref: https://huggingface.co/LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct
res = "exaone"
+ if chkhsh == "1baddeb572cd9de2a6d36f2ad0c361490bf5447dafca20afbac625e9d37f18a5":
+ res = "llama-bpe"
if res is None:
logger.warning("\n")
But from my quick check it looks like the model doesn't work:
(llama.cpp) phm@epyc:~/projects/llama.cpp$ ./llama-minicpmv-cli -m models/MiniCPM-V-2_5/Model-8.0B-F16.gguf --mmproj models/MiniCPM-V-2_5/mmproj-model-f16.gguf -c 4096 --temp 0.7 --top-p 0.8 --top-k 100 --repeat-penalty 1.05 --image /home/phm/Downloads/demo-2.jpg -p "What is in the image?"
Log start
clip_model_load: description: image encoder for MiniCPM-V
clip_model_load: GGUF version: 3
clip_model_load: alignment: 32
clip_model_load: n_tensors: 455
clip_model_load: n_kv: 19
clip_model_load: ftype: f16
clip_model_load: loaded meta data with 19 key-value pairs and 455 tensors from models/MiniCPM-V-2_5/mmproj-model-f16.gguf
clip_model_load: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
clip_model_load: - kv 0: general.architecture str = clip
clip_model_load: - kv 1: clip.has_text_encoder bool = false
clip_model_load: - kv 2: clip.has_vision_encoder bool = true
clip_model_load: - kv 3: clip.has_minicpmv_projector bool = true
clip_model_load: - kv 4: general.file_type u32 = 1
clip_model_load: - kv 5: general.description str = image encoder for MiniCPM-V
clip_model_load: - kv 6: clip.projector_type str = resampler
clip_model_load: - kv 7: clip.minicpmv_version i32 = 3
clip_model_load: - kv 8: clip.vision.image_size u32 = 448
clip_model_load: - kv 9: clip.vision.patch_size u32 = 14
clip_model_load: - kv 10: clip.vision.embedding_length u32 = 1152
clip_model_load: - kv 11: clip.vision.feed_forward_length u32 = 4304
clip_model_load: - kv 12: clip.vision.projection_dim u32 = 0
clip_model_load: - kv 13: clip.vision.attention.head_count u32 = 16
clip_model_load: - kv 14: clip.vision.attention.layer_norm_epsilon f32 = 0.000001
clip_model_load: - kv 15: clip.vision.block_count u32 = 26
clip_model_load: - kv 16: clip.vision.image_mean arr[f32,3] = [0.500000, 0.500000, 0.500000]
clip_model_load: - kv 17: clip.vision.image_std arr[f32,3] = [0.500000, 0.500000, 0.500000]
clip_model_load: - kv 18: clip.use_gelu bool = true
clip_model_load: - type f32: 285 tensors
clip_model_load: - type f16: 170 tensors
clip_model_load: CLIP using CPU backend
clip_model_load: text_encoder: 0
clip_model_load: vision_encoder: 1
clip_model_load: llava_projector: 0
clip_model_load: minicpmv_projector: 1
clip_model_load: model size: 1044.86 MB
clip_model_load: metadata size: 0.16 MB
clip_model_load: params backend buffer size = 1044.86 MB (455 tensors)
key clip.vision.image_grid_pinpoints not found in file
key clip.vision.mm_patch_merge_type not found in file
key clip.vision.image_crop_resolution not found in file
clip_image_build_graph: 448 448
ggml/src/ggml.c:4484: GGML_ASSERT(ggml_can_repeat(b, a)) failed
Could not attach to process. If your uid matches the uid of the target
process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
again as the root user. For more details, see /etc/sysctl.d/10-ptrace.conf
ptrace: Operation not permitted.
No stack.
The program is not being run.
Aborted (core dumped)
I don't know, maybe adding support for MiniCPM-V2.6 broke support for MiniCPM-V2.5.
I think I found the culprit, this line overrides minicpmv_version value set in the command line (--minicpmv_version 2
), remove it and everything starts working correctly:
diff --git a/examples/llava/minicpmv-convert-image-encoder-to-gguf.py b/examples/llava/minicpmv-convert-image-encoder-to-gguf.py
index ea773742..b44d1082 100644
--- a/examples/llava/minicpmv-convert-image-encoder-to-gguf.py
+++ b/examples/llava/minicpmv-convert-image-encoder-to-gguf.py
@@ -587,7 +587,6 @@ elif args.minicpmv_projector is not None:
fname_middle = "mmproj-"
has_text_encoder = False
has_minicpmv_projector = True
- minicpmv_version = 3
elif args.vision_only:
fname_middle = "vision-"
has_text_encoder = False
The command line option --minicpmv_version 2
is specified in the README here:
What happened?
Follow the steps in README-minicpmv2.5.md#usage to convert
minicpm v2.5
. The conversion process fails while running the command:python ./convert_hf_to_gguf.py ../MiniCPM-Llama3-V-2_5/model
. Specifically, the error happened after inputy
to answer the questionDo you wish to run the custom code? [y/N]
.Name and Version
version: 3604 (1b6ff90f) built with cc (Ubuntu 11.2.0-19ubuntu1) 11.2.0 for x86_64-linux-gnu
What operating system are you seeing the problem on?
Linux
Relevant log output