search

vikhyat / moondream

tiny vision language model
https://moondream.ai
Apache License 2.0
5.14k stars 447 forks source link

Fine tune the model with custom dataset #127

Open Sunnyburnwal01123 opened 2 months ago

Sunnyburnwal01123 commented 2 months ago

Can I fine tune the model with the images using coordinates to detect the object? If it is possible, how can achieve it? At what image size the model take to fine tune with?

vikhyat commented 2 months ago

This will be coming in the next release, around Aug 19.

On Sat, Aug 10, 2024 at 02:17 Sunnyburnwal01123 @.***> wrote:

Can I fine tune the model with the images using coordinates to detect the object? If it is possible, how can achieve it? At what image size the model take to fine tune with?

— Reply to this email directly, view it on GitHub https://github.com/vikhyat/moondream/issues/127, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAHSV6BNJ4X52REENPW6ETZQXLEFAVCNFSM6AAAAABMJVUBWWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ2TSMBQGI2DENQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

Sunnyburnwal01123 commented 2 months ago

Thanks for your response.

Sunnyburnwal01123 commented 2 months ago

But I am also facing issue i.e process is getting killed while saving the fine tuned model that is fine tuned with 200 images with resolution 1920 x 1080 . But when I save its checkpoint using torch (where I have to load base model as well as checkpoints that consuming double memory for inferencing ) or reduced the image size to 720 x 407, then the model is saving.

Memory - 16 GB Gpu - 8 GB