How can I provide some examples to BLIP2 and InstructBLIP models?

salesforce / LAVIS

LAVIS - A One-stop Library for Language-Vision Intelligence

BSD 3-Clause "New" or "Revised" License

9.94k stars 972 forks source link

How can I provide some examples to BLIP2 and InstructBLIP models? #748

Open xukefaker opened 1 month ago

xukefaker commented 1 month ago

I am trying to improve the performance of BLIP2 and InstructBLIP models by giving some pre-defined examples (user-assistant chat history), how can I do this?

xukefaker commented 1 month ago

Like: messages.append( { "role": "user", "content": f"...." } ) messages.append( { "role": "assistant", "content": f"...." } )