search

autodistill / autodistill-kosmos-2

Kosmos-2 base model for use with Autodistill.
https://docs.autodistill.com
MIT License
5 stars 2 forks source link

No output #1

Open shersoni610 opened 11 months ago

shersoni610 commented 11 months ago

Hello, I tried Kosmos-2 with some high-quality images: The code generated images and labels files in the folder. but all the labels were empty. Here is the code that I was running on Apple M1. What could go wrong?

from autodistill_kosmos_2 import Kosmos2 from autodistill.detection import CaptionOntology import sys base_model = Kosmos2(ontology=CaptionOntology({"person": "person"})) folder_name = sys.argv[1] base_model.label(input_folder=folder_name, output_folder='labels')

docmarionum1 commented 8 months ago

I was having the same issue. The ontology label needs to match the output from Kosmos2 very closely. For example, if it does generate an entity for a person in your examples, the entity might be tagged "a person" or "the person" in which case it wouldn't match what you provided. Even capitalization would interfere with a match based on how this is currently coded.

In general, this doesn't feel like a good model for using as a base for autodistill. Even with the ability to change the prompt (which isn't exposed right now), I haven't been able to get it to give consistent results.