keras-team / keras-cv

Industry-strength Computer Vision workflows with Keras
Other
1k stars 333 forks source link

Not able to run `CLIP Model` using `from_present` #2472

Open carrycooldude opened 1 month ago

carrycooldude commented 1 month ago

Current Behavior:

Screenshot from 2024-08-06 21-21-59

Expected Behavior:

not able to load CLIP model even though I installed keras-nlp too

Steps To Reproduce:

Screenshot from 2024-08-06 21-23-59

Version:

Screenshot from 2024-08-06 21-25-22

Anything else:

This is my colab link

sachinprasadhs commented 1 month ago

You need to install the dependencies and load the clip model as below.

!pip install -U keras-cv
!pip install -U keras-nlp
!pip install -U keras

import tensorflow as tf
from tensorflow.keras.layers import Input, Dense, Concatenate
from tensorflow.keras.models import Model
from keras_cv.models import CLIP
import numpy as np

# Load the pre-trained CLIP model from KerasCV
clip_model = CLIP.from_preset("clip-vit-base-patch16")

for Image and Text encoder, you can use as below.

# Define inputs
image_input = Input(shape=(224, 224, 3), name='image_input')
text_input = Input(shape=(77,), name='text_input', dtype=tf.int32)

# Process image through CLIP
image_embeds = clip_model.image_encoder(image_input)

# Process text through CLIP
text_embeds = clip_model.text_encoder(text_input)
github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

carrycooldude commented 3 weeks ago

thanks @sachinprasadhs , it's working but I guess it's not in docs Can I contribute to it?

sachinprasadhs commented 1 week ago

Sure