Custom Diffusion retrieve.py

[rezkanas]

Describe the bug

when running retrieve.py to produce class prior photo using laion datasset,, I run through below errors. url is bad in the code below... could be the root cause?

client = ClipClient(
        url="https://knn.laion.ai/knn-service", indice_name="laion5B-L-14", num_images=num_images, aesthetic_weight=0.1
    )

Reproduction

!python retrieve.py --class_prompt person --class_data_dir /mnt/c/Users/noobw/PycharmProjects/pythonProject/custom_diffusion_anas/class_prior --num_class_images 200

Logs

/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Traceback (most recent call last):
  File "/home/anasrezklinux/.local/lib/python3.10/site-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/anasrezklinux/test_pycharm_link/diffusers/examples/custom_diffusion/retrieve.py", line 87, in <module>
    retrieve(args.class_prompt, args.class_data_dir, args.num_class_images)
  File "/home/anasrezklinux/test_pycharm_link/diffusers/examples/custom_diffusion/retrieve.py", line 37, in retrieve
    class_images = client.query(text=class_prompt)
  File "/home/anasrezklinux/.local/lib/python3.10/site-packages/clip_retrieval/clip_client.py", line 84, in query
    return self.__search_knn_api__(text=text)
  File "/home/anasrezklinux/.local/lib/python3.10/site-packages/clip_retrieval/clip_client.py", line 153, in __search_knn_api__
    ).json()
  File "/home/anasrezklinux/.local/lib/python3.10/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

System Info

• diffusers version: 0.26.1
• Platform: Linux-5.15.133.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
• Python version: 3.10.12
• PyTorch version (GPU?): 2.2.0+cu121 (True)
• Huggingface_hub version: 0.20.3
• Transformers version: 4.37.0
• Accelerate version: 0.25.0
• xFormers version: 0.0.24
• Using GPU in script?:
• Using distributed or parallel set-up in script?:

Who can help?

@sayakpaul @patrickvonplaten

[Ashutosh7982]

The website "https://knn.laion.ai/knn-service is down

[rezkanas]

I realize that but then how can retrieve.py be run?

[sayakpaul]

Meaning?

[rezkanas]

@sayakpaul, I don't understand your response. website is down, do I need to wait? or probably the url is broken?

[OchiaiNao]

You can add these images manually with the following structure:

your_class_data_dir
├── images
│   ├── 01.jpg
│   ├── 02.jpg
├── caption.txt # captions line by line
└── images.txt # absolute path line by line

Also, remember the number of images should be the same as:

--num_class_images=200

[sayakpaul]

The website incurs ratelimiting issues because it's used by many folks. So, that is a constraint.

[yiyixuxu]

It this fixed?

[sayakpaul]

@yiyixuxu it depends on the service that retrieve.py relies on. We cannot do much here.

[TruongQuocTruong]

I also has the same issue

[daeunni]

Same error. Do u guys can run retrieve.py now?

[WuTao-CS]

I have the same issue,

[sayakpaul]

The retrieve.py script relies on external services. Commenting with the same thing won’t help the issue :-)

[Sakura-hub47]

I have the same issue, are there any alternative URLs available?

[sayakpaul]

I don't think there is.

[boringKey]

I don't think there is.

Hi, can you share a new link which includes the data (from retrieve.py) we want? So sorry to bother you.

[asomoza]

FYI as people doesn't seem to know:

retrieve.py is from clip-retrieval which connected to the LAION dataset.

LAION has a zero tolerance policy for illegal content and in an abundance of caution, we are temporarily taking down the LAION datasets to ensure they are safe before republishing them.

This was posted on the 19th of December, 2023 and it still down, so don't expect it to be up soon if it ever comes back.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[sayakpaul]

retrieve.py is not within our control (diffusers). So, will close this.