OpenBMB / MiniCPM-V

MiniCPM-V 2.6: A GPT-4V Level MLLM for Single Image, Multi Image and Video on Your Phone
Apache License 2.0
11.88k stars 836 forks source link

Inference after lora finetuning #557

Closed tristan279 closed 2 days ago

tristan279 commented 1 week ago

i finetuned miniCPM-v-2_6 with this dataset format:

"id":str(i), "image":{"<image_upper>":f"path_upper","<image_lower>":f"path_lower"}, "conversations": [{"role": "user", "content": f"<image_upper><image_lower>{prompt}"},{"role":"assistant","content":text}]

how do i invoke the model with the exact same prompt, using transformers library? prompt format was not given in the finetuning example

LDLINGLINGLING commented 1 week ago

https://modelbest.feishu.cn/wiki/HvfLwYzlIihqzXkmeCdczs6onmd?from=from_copylink,please check this blog