Instructions to train / fine tune on our own data

[IamShubhamGupto]

Hey

Thank you for releasing nanoowl, I think it's really helpful for my ongoing work. Is there a way to fine-tune the weights for my own data?

Instructions on how train / fine tune would be great!

Thank you

[jaybdub]

Hey @IamShubhamGupto ,

Thanks for reaching out!

We don't have this feature at the moment, but I'll update this thread if that changes.

Depending on your use case, you might be able to provide image embeddings instead of text embeddings for querying objects. We haven't implemented this yet either though :sweat_smile: .

Let me know if you have any questions, or anything else I can do to help.

John

[IamShubhamGupto]

Hey @jaybdub

Thank you for the feedback! Understood, I'll look into it in my own time as well but essentially we wanted to tackle very niche use cases for object detection using Nanoowl (as training our own model is painful).

For now I guess prompt engineering is the way to go

[elfar]

Hi @jaybdub - truly awesome stuff! Any plans on adding image embeddings you mentioned in your comment as an option as well. E.g. selecting a bounding box of something in image A and looking for the image within that selection in image B? Alternatively could you roughly point me in the right direction if I found the time to look into implementing that feature myself!

-elfar

[Aki1991]

Hi @jaybdub,

Is there any update on the training of this model on custom dataset? This model is used in metropolis, so It would be incredibly helpful to have this implemented.

As another option, I have trained the model from the official Owl-ViT repo but here the model from transformer have been used. I tried using that model to convert to .engine file but it is not working. Is there a way to build an .engine file from that model from official repo?

-Akash