How can I use CLIP model to obtain the image and text embeddings?

[zhujiajian98]

Now I need to try to use the prompt training scheme to finetune the yolo-world. But according to the docs/prompt_yolo_world.md, I need to extract the image embedding/text embedding by using the clip model and save the corresponding npy file. So now how can I use CLIP model to obtains the image and text embeddings?

[wondervictor]

Hi @zhujiajian98, text embeddings can be extracted by generate_text_prompts.py, I'll provide the scripts for image embeddings in a day (before the next Monday).

[liping-ren]

Hello Author, I am interested in using images for fine tuning and would like to ask if you can provide the script that generates the image embedding? Thank you very much.

[wondervictor]

Hi @liping-ren, it has been added to tools/generate_image_prompts.py.