search

CircleRadon / Osprey

[CVPR2024] The code for "Osprey: Pixel Understanding with Visual Instruction Tuning"
Apache License 2.0
775 stars 42 forks source link

How to use Osprey in my own dataset #22

Closed joy91269 closed 9 months ago

joy91269 commented 9 months ago

Thank you for your amazing work. And here is the thing, what should i do to link your Osprey with my own dataset? I have some images collected by our team. Should i retrain the model to make the description of every area more accurate? how to do this? And because it is a folder of images not just one image that i can use in the demo. How to use your model in a large amount of images and collect all the result?

CircleRadon commented 9 months ago

Hello, @joy91269 If you find that the generated description of our model is not so accurate for your domain data, you can consider finetuning the model with your own dataset. To do this, ensure your data is formatted in accordance with the osprey-724k dataset we've provided.

When you wish to conduct inference on a collection of images, you'll need to run the demo code on your local machine. Additionally, you should craft a script that processes each image, feeding the each mask region into the model to obtain the corresponding answers.

joy91269 commented 9 months ago

I send you a email to your @zju.edu.com to ask more detail things.