Creating custom use cases based on specific images.

Vision-CAIR / MiniGPT-4

Open-sourced codes for MiniGPT-4 and MiniGPT-v2 (https://minigpt-4.github.io, https://minigpt-v2.github.io/)

https://minigpt-4.github.io

BSD 3-Clause "New" or "Revised" License

25.4k stars 2.91k forks source link

Creating custom use cases based on specific images. #416

Open smtabatabaie opened 11 months ago

smtabatabaie commented 11 months ago

Very awesome project, thanks very much. I wanted to ask if it is possible to provide special images for and make the model train on those? for example, creating a manual for a specific tool or machine based on additional visual data. Kinda like RAG but for images. Thanks

junchen14 commented 11 months ago

you can finetune our model. There are some finetuning examples in here

Crocodile-Chris commented 11 months ago

you can finetune our model. There are some finetuning examples in here Is it possible to finetune MiniGPT-v2 for specific use cases? If so, should I use the stage 2 or stage 3 checkpoint provided by you guys? (I assume I have to make my own dataset) Thank you!