web demo

[AK391]

Hi, thanks for releasing the code and models, would you be interested in creating a web demo for CogView2 using Gradio on Hugging Face?

The Hub offers free hosting, and it would make your work more accessible and visible to the rest of the ML community. Models/datasets/spaces(web demos) can be added to a user account or organization similar to github.

here is a example Gradio Demo for dalle-mini: https://huggingface.co/spaces/dalle-mini/dalle-mini

and another example that for a cvpr 2022 paper: https://huggingface.co/spaces/CVPR/ml-talking-face

and here is a guide for adding web demo to the organization: https://huggingface.co/blog/gradio-spaces

Please let us know if you would be interested and if you have any questions, we can also help with the technical implementation.

[zesameri]

I believe this link (https://agc.platform.baai.ac.cn/CogView/index.html)

[vvvm23]

@zesameri is that not for CogView rather than CogView2?

[Sleepychord]

@vvvm23 The web demo is only slightly different from the cogview2 in the repo, we will quickly update it.

[Sleepychord]

@AK391 but CogView2 is super large, I will look into it but not sure it is okay.

[julien-c]

in case you would be interested in hosting a version of the demo app on HuggingFace (hf.co) we would be super happy to support you!

[AK391]

@Sleepychord we should be able to add large models to the hub, see https://huggingface.co/docs/hub/models-uploading for how to add models

[kamalkraj]

https://github.com/kamalkraj/CogView2/tree/gradio -Check readme

Simple gradio interface.

We can also run the backend alone using the replicate docker image

docker run -d -p 5000:5000 --gpus=all r8.im/thudm/cogview2@sha256:8bd18a43f9113352683ea9fe70945e8150ad2b60bdb109ce74832e8b6c528d68

And connect a gradio app like below.

import base64
import json
from io import BytesIO

import gradio as gr
import requests
from PIL import Image

url = "http://localhost:5000/predictions"

headers = {"Content-Type": "application/json"}

def image_grid(imgs, rows, cols):
    assert len(imgs) == rows * cols

    w, h = imgs[0].size
    grid = Image.new("RGB", size=(cols * w, rows * h))

    for i, img in enumerate(imgs):
        grid.paste(img, box=(i % cols * w, i // cols * h))
    return grid

def postprocess(data):
    images = []
    for image_data in data["output"]:
        img_str = image_data["image"][22:]
        im = Image.open(BytesIO(base64.b64decode(img_str)))
        images.append(im)
    return image_grid(images, 4, 4)

def predict(text, style):
    payload = json.dumps({"input": {"text": text, "style": style}})
    response = requests.request("POST", url, headers=headers, data=payload)
    data = response.json()
    return postprocess(data)

gr.Interface(
    fn=predict,
    inputs=[
        "text",
        gr.Dropdown(
            choices=[
                "none",
                "mainbody",
                "photo",
                "flat",
                "comics",
                "oil",
                "sketch",
                "isometric",
                "chinese",
                "watercolor",
            ],
            value="none",
        ),
    ],
    outputs="image",
).launch()

[kamalkraj]

[Sleepychord]

@kamalkraj @AK391 Looks great! but I don't want to pretend to know how gradio works lol... So maybe you can

1. give a slightly detailed steps in readme.
2. make sure the --gradio will not affect other effects (maybe need a new separated python file?).
3. pull request
4. maybe a button to control temp_all could be better to control the extend of randomness? Maybe we can work to push it to /gradio-spaces

[AK391]

@kamalkraj @Sleepychord Great work on the Gradio Demo, we are looking to host it on Hugging Face Spaces, feel free to join the organization here https://huggingface.co/THUDM, by clicking the request to join organization button and pushing the gradio demo to https://huggingface.co/spaces/THUDM/CogView2, similar to github the demo can be managed and worked on in a collaborative repo, thanks

[AK391]

@Sleepychord opened a PR for the Gradio Web Demo https://github.com/THUDM/CogView2/pull/19

[Sleepychord]

Hi, @AK391 I have merged the pull request! Thank you for your work!