Open ganliqiang opened 4 months ago
What GPU model are you using?
What GPU model are you using?
a100 80g,the complete is from lmdeploy import pipeline, ChatTemplateConfig, GenerationConfig from lmdeploy.vl import load_image
gen_config = GenerationConfig(top_k=1, temperature=0)
pipe = pipeline('liuhaotian/llava-v1.5-13b',
chat_template_config=ChatTemplateConfig(model_name='vicuna'), cache_max_entry_count=0.1)
image = load_image(img_path)
vqa_time = time.time()
response = pipe((prompt, image), gen_config=gen_config)
print(f"vqa_tiem:{time.time() - vqa_time}")
print(response)
ans=find_ans(response.text)
print(ans)
i find most time the gpu is idle,so i guess maybe most of time it is preprocessing the image?
Motivation
Related resources
No response
Additional context
No response