search

pharmapsychotic / clip-interrogator

Image to prompt with BLIP and CLIP
MIT License
2.58k stars 429 forks source link

Naming specific colors with training #88

Open frazras opened 11 months ago

frazras commented 11 months ago

I have an application that I am curious to know if this is the correct tool.

I am in the balloon decoration niche and I want to be able to determine the colors used in a balloon arrangement like this: https://imgur.com/q41Np4i

I want to train it on the name of specific balloon colors and have it tell me the name based on the provided image. For example the colors in the image above might be: Buttercream, peach pastel and soft blue.

Is it possible to get those results from this tool with the correct training and configuration?

pharmapsychotic commented 10 months ago

hmm if you have the data you could train a classification head on top of CLIP image embedding. this isn't something CLIP Interrogator has, it just makes use of the pretrained OpenAI and open_clip models.