rom1504 / clip-retrieval

Easily compute clip embeddings and build a clip retrieval system with them
https://rom1504.github.io/clip-retrieval/
MIT License
2.42k stars 213 forks source link

How to visualize images from the API results? (webdataset format) #241

Closed JJumSSu closed 10 months ago

JJumSSu commented 1 year ago

Hi! First of all, thanks for the awesome repo!

I'm currently trying to deploy my own clip-retrieval API using my own data (stored in the local machine). The dataset is in webdataset format, which I used to make and call an API.

I managed to get a retrieval result using the API, but am trying to figure out how to visualize the image.

For example, from the responses,

[
    {
        "caption": "A small white dog wearing a black hat. ",
        "image_path": "00001153",
        "id": 108,
        "similarity": 0.4931468367576599
    },
    {
        "caption": "A lit candle in the shape of an elephant.",
        "image_path": "00001688",
        "id": 1152,
        "similarity": 0.4843543767929077
    },
  ...
]

I would like to visualize the images. But since they are stored in .tarfiles (webdatset format), I can not visualize them. Are there any ways to visualize the retrieved results?

Thank you!

rom1504 commented 1 year ago

webdataset does not support random access

I think the simplest solution is for you to put your images under a simple http static server and provide urls to clip retrieval