Internal Server Error while searching by text

[vishaal27]

Hi,

I am trying to search the hosted clip-retrieval backend index by text and end up getting an internal server error. Here is the code I used:

from clip_retrieval.clip_client import ClipClient, Modality
client = ClipClient(url="https://knn.laion.ai/knn-service", indice_name="laion5B-L-14", modality=Modality.TEXT)
results = client.query(text="an image of a cat")
print(results)

Output:

{'message': 'Internal Server Error'}

However, when I search by image it seems to be working fine:

from clip_retrieval.clip_client import ClipClient, Modality
client = ClipClient(url="https://knn.laion.ai/knn-service", indice_name="laion5B-L-14", modality=Modality.IMAGE)
results = client.query(text="an image of a cat")
print(results[0])

Output:

{'caption': 'orange cat with supicious look stock photo', 'url': 'https://media.istockphoto.com/photos/orange-cat-with-supicious-look-picture-id907595140?k=6&m=907595140&s=612x612&w=0&h=4CTvSxNvv4sxSCPxViryha4kAjuxDbrXM5vy4VPOuzk=', 'id': 518836491, 'similarity': 0.5591729879379272}

Please let me know if I am doing something incorrect, or if direct text-to-text similarity search is not enabled by default in clip-retrieval's client? Thanks :)

[rom1504]

Laion5B text index is not computed indeed but you can use laion400m one it you want

What's your use case ?

On Mon, Sep 11, 2023, 12:11 Vishaal Udandarao @.***> wrote:

Hi,

I am trying to search the hosted clip-retrieval backend index by text and end up getting an internal server error. Here is the code I used:

from clip_retrieval.clip_client import ClipClient, Modalityclient = ClipClient(url="https://knn.laion.ai/knn-service", indice_name="laion5B-L-14", modality=Modality.TEXT)results = client.query(text="an image of a cat")print(results)

Output:

{'message': 'Internal Server Error'}

However, when I search by image it seems to be working fine:

from clip_retrieval.clip_client import ClipClient, Modalityclient = ClipClient(url="https://knn.laion.ai/knn-service", indice_name="laion5B-L-14", modality=Modality.IMAGE)results = client.query(text="an image of a cat")print(results[0])

Output:

{'caption': 'orange cat with supicious look stock photo', 'url': 'https://media.istockphoto.com/photos/orange-cat-with-supicious-look-picture-id907595140?k=6&m=907595140&s=612x612&w=0&h=4CTvSxNvv4sxSCPxViryha4kAjuxDbrXM5vy4VPOuzk=', 'id': 518836491, 'similarity': 0.5591729879379272}

Please let me know if I am doing something incorrect, or if direct text-to-text similarity search is not enabled by default in clip-retrieval's client? Thanks :)

— Reply to this email directly, view it on GitHub https://github.com/rom1504/clip-retrieval/issues/311, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAR437RHNIBH4NT5HCC6ITTXZ3IV7ANCNFSM6AAAAAA4S7ZMDY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[vishaal27]

My main usecase: I want to do a text search for certain keywords in all the captions of the LAION-5B dataset, what would you recommend is the fastest way to do this (without locally downloading the parquet files)? I realise the clip index does not do direct text matching but rather cosine similarities. I would still be interested in pure text-to-text matching on the LAION-5B index. For example, I want to search "gaussian noise" as keyword and check for exact matches / most similar text captions in the LAION-5B dataset.