gwang-kim / DiffusionCLIP

[CVPR 2022] Official PyTorch Implementation for DiffusionCLIP: Text-guided Image Manipulation Using Diffusion Models
Other
785 stars 113 forks source link

Add Replicate demo and API #15

Closed chenxwh closed 2 years ago

chenxwh commented 2 years ago

Hey @gwang-kim ! 👋

This pull request makes it possible to run your model inside a Docker environment, which makes it easier for other people to run it. We're using an open source tool called Cog to make this process easier.

This also means we can make a web page where other people can run your model! We have added some of the pre-trained models to the demo, view it here: https://replicate.com/cjwbw/diffusionclip

Replicate also have an API, so people can easily run your model from their code:

import replicate
model = replicate.models.get("cjwbw/diffusionclip")
model.predict(image="...")

If you'd like to modify the Replicate page (e.g. Example Gallery), let me know and I can transfer ownership to your account.

In case you're wondering who I am, I'm from Replicate, where we're trying to make machine learning reproducible. We got frustrated that we couldn't run all the really interesting ML work being done. So, we're going round implementing models we like. 😊

gwang-kim commented 2 years ago

Hi, @chenxwh! Thanks for making great demo of our work! I checked the demo and it was amazing! However, I think I need to change some code for the face image manipulation to enable align_face function to align random face images. Can I modify the demo code?

I signed in replicate with github, so my ID is @gwang-kim.

Thanks in advance!

chenxwh commented 2 years ago

Hi @gwang-kim,

Of course, we just wanted to demonstrate how the models can be hosted on Replicate, and the authors can modify and update it however they like :) Here is your model page now https://replicate.com/gwang-kim/diffusionclip.

With cog login and cog push r8.im/gwang-kim/diffusionclip you will be able to push the changes to the demo site. More documentation can be found here.

gwang-kim commented 2 years ago

Hi, @chenxwh!

Pushing image 'r8.im/gwang-kim/diffusionclip'...
Using default tag: latest
The push refers to repository [r8.im/gwang-kim/diffusionclip]
4d53debfd03c: Preparing 
53407a2e1584: Preparing 
a4190acdd3ec: Preparing 
29739987312b: Preparing 
87715f84e3b2: Preparing 
55bc2625d57b: Waiting 
648152932fbd: Waiting 
fe3c7e68a3ca: Waiting 
59d4c0cceaa4: Waiting 
1b3522040815: Waiting 
8b455a82f26c: Waiting 
6dcc9b300b40: Waiting 
80e16222d806: Waiting 
09fd1d816e84: Waiting 
92049436f54b: Waiting 
526ad5995ec8: Waiting 
12295fb6bd46: Waiting 
d2a2f6974192: Waiting 
321f6b26846b: Waiting 
51d06c14f508: Waiting 
b863c622341d: Waiting 
95129a5fe07e: Waiting 
unauthorized: You don't have write access to this model
â…¹ exit status 1

Could you tell me how I can fix unauthorized: You don't have write access to this model issue?

chenxwh commented 2 years ago

hi have you cog login first? it will sign you in and grant you the access to model under you account

gwang-kim commented 2 years ago

@chenxwh I fix the authorization issue. Thanks! I have another question! Can I pull or fetch the docker (cog) image from the replicate server? If possible, it will make it easier to modify the code.

chenxwh commented 2 years ago

Hi @gwang-kim,

Yes you can find the image-uri under "Run on your own computer" tab from the web demo, and then:

docker pull <image-uri>
docker run --name=my-image -d --entrypoint sleep <image-uri> infinity  # start the image in the background and have it sleep
docker cp my-image:/src <destination-folder>
docker kill my-image; docker rm my-image  # clean up

The image-uri in your case is r8.im/gwang-kim/diffusionclip@sha256:f10ee426c918603cb8e9d56542f60ccfc21eea1c5b2d7fa663441e2334fdb68f

Let me know if you need further clarification :)

gwang-kim commented 2 years ago

Thanks for your kind reply, @chenxwh!

I updated the demo following your instructions!