OpenBMB / llama.cpp

Port of Facebook's LLaMA model in C/C++
MIT License
37 stars 7 forks source link

推理MiniCPM-V2失败 #14

Open helloheshee opened 1 month ago

helloheshee commented 1 month ago

What happened?

作者您好,我基于分支minicpm-v2.5构建了应用程序,MiniCPM-V-2.5 模型运行良好。 从 Hugging Face 上下载了 MiniCPM-V-2-GGUF文件,运行失败,请问是不支持 MiniCPM-V2 吗?

Name and Version

./bin/minicpmv-cli -m /home/wuzhi/share/models/llama/MiniCPM-V-2-GGUF/MiniCPM-V-2.Q4_0.gguf --mmproj /home/wuzhi/share/models/llama/MiniCPM-V-2-GGUF/MiniCPM-V-2-mmproj.F16.gguf --image /home/wuzhi/share/projects/openBMB/640x360_frame.jpg -p "图片里有什么?"

What operating system are you seeing the problem on?

Linux

Relevant log output

Log start
clip_model_load: description:  image encoder for LLaVA
clip_model_load: GGUF version: 3
clip_model_load: alignment:    32
clip_model_load: n_tensors:    440
clip_model_load: n_kv:         18
clip_model_load: ftype:        f16

clip_model_load: loaded meta data with 18 key-value pairs and 440 tensors from /home/wuzhi/share/models/llama/MiniCPM-V-2-GGUF/MiniCPM-V-2-mmproj.F16.gguf
clip_model_load: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
clip_model_load: - kv   0:                       general.architecture str              = clip
clip_model_load: - kv   1:                      clip.has_text_encoder bool             = false
clip_model_load: - kv   2:                    clip.has_vision_encoder bool             = true
clip_model_load: - kv   3:                   clip.has_llava_projector bool             = true
clip_model_load: - kv   4:                          general.file_type u32              = 1
clip_model_load: - kv   5:                        general.description str              = image encoder for LLaVA
clip_model_load: - kv   6:                        clip.projector_type str              = resampler
clip_model_load: - kv   7:                     clip.vision.image_size u32              = 448
clip_model_load: - kv   8:                     clip.vision.patch_size u32              = 14
clip_model_load: - kv   9:               clip.vision.embedding_length u32              = 1152
clip_model_load: - kv  10:            clip.vision.feed_forward_length u32              = 4304
clip_model_load: - kv  11:                 clip.vision.projection_dim u32              = 0
clip_model_load: - kv  12:           clip.vision.attention.head_count u32              = 16
clip_model_load: - kv  13:   clip.vision.attention.layer_norm_epsilon f32              = 0.000001
clip_model_load: - kv  14:                    clip.vision.block_count u32              = 26
clip_model_load: - kv  15:                     clip.vision.image_mean arr[f32,3]       = [0.500000, 0.500000, 0.500000]
clip_model_load: - kv  16:                      clip.vision.image_std arr[f32,3]       = [0.500000, 0.500000, 0.500000]
clip_model_load: - kv  17:                              clip.use_gelu bool             = true
clip_model_load: - type  f32:  277 tensors
clip_model_load: - type  f16:  163 tensors
clip_model_load: CLIP using CPU backend
GGML_ASSERT: /home/wuzhi/share/projects/openBMB/llama.cpp/examples/minicpmv/clip.cpp:1188: new_clip->has_minicpmv_projector
没有符号表被读取。请使用 "file" 命令。
Could not attach to process.  If your uid matches the uid of the target
process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
again as the root user.  For more details, see /etc/sysctl.d/10-ptrace.conf
ptrace: 不允许的操作.
No stack.
The program is not being run.
已放弃 (核心已转储)
aaa123-t commented 2 weeks ago

请问你基于分支minicpm-v2.5构建应用程序时,推理速度快吗,我这里推理好慢,大概1token/s