Zhouziyi828 commented 1 month ago

看到了支持单个图片调用模型进行推理,是否支持批量的推理?

LDLINGLINGLING commented 1 month ago

可以使用LMdeploy,代码如下 from lmdeploy import pipeline, TurbomindEngineConfig from lmdeploy.vl import load_image

可将MiniCPM-Llama3-V 2.5换成本地路径

session_len=2048 代表上下文长度

tp=8 代表使用显卡数，必须是2**n，比如1，2，4，8

pipe = pipeline('MiniCPM-Llama3-V 2.5', backend_config=TurbomindEngineConfig(session_len=2048,tp=8),)

图片的url或者本地路径

image_urls=[ "/root/ld/ld_project/MiniCPM-V/assets/minicpmv2-cases.png", "/root/ld/ld_project/MiniCPM-V/assets/llavabench_compare_phi3.png", "/root/ld/ld_project/MiniCPM-V/assets/MiniCPM-Llama3-V-2.5-peformance.png", "/root/ld/ld_project/MiniCPM-V/assets/zhihu.webp", "/root/ld/ld_project/MiniCPM-V/assets/thunlp.png" ] prompts = [('describe this image', load_image(img_url)) for img_url in image_urls] response = pipe(prompts) print([i.text for i in response])

Zhouziyi828 commented 1 month ago

我用这个代码会出现: Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. [WARNING] gemm_config.in is not found; using default GEMM algo
[WARNING] gemm_config.in is not found; using default GEMM algo Segmentation fault (core dumped) 然后没有任何输出