How can I create a Flask app that runs ComfyScript code?

[pranaykoppula]

For the life of me, I can't get this to work:

from flask import Flask, request, Response, jsonify

app = Flask(__name__)
from comfy_script.runtime import *
load(args=ComfyUIArgs('--force-fp16'))
from comfy_script.runtime.nodes import *

def comfy_function(prompt:str):
    with Workflow():
[. Some ComfyScript code ...]

@app.route("/generate",methods=['POST'])
def genimages()
    data = request.get_json()
    usr_prompt=data.get('prompt')
    res=comfy_function(usr_prompt)
    response = jsonify(res)
    return response

if __name__ == "__main__":
    app.run(host = 'X.X.X.X', port= XXXX)

I don't know what I'm doing wrong. Comfy seems to start, but when I send a request to the API, nothing happens. Is this simply not possible?

[Chaoses-Ib]

How do you wait for the workflow and get the results in [. Some ComfyScript code ...]?

[pranaykoppula]

I thought that could work synchronously. I mean even if that is the issue, when I try to print something before I call the comfy_function, it doesn't print anything. But the comfy server seems to start

[pranaykoppula]

This is how the function looks:

def imagegen(prompt_text="The Wild West"):

    promptlist = promptgen(prompt_text,'image')

    print(promptlist)

    image_library=[]
    with Workflow:
        maskim=PIL.Image.open('/root/Pranay/SlotMachineDemo/ComfyUI/input/circle_mask.png')
        refim=PIL.Image.open('/root/Pranay/SlotMachineDemo/ComfyUI/input/styleref.jpg')
        model, clip, vae = CheckpointLoaderSimple('DreamShaper_8_pruned.safetensors')
        ipadapter = IPAdapterModelLoader('ip-adapter-plus_sd15.bin')
        image, _ = LoadImage(refim)
        image = PrepImageForClipVision(image, 'LANCZOS', 'top', 0)
        image2, _ = LoadImage(maskim)
        mask = ImageToMask_(image2, 'intensity')
        clip_vision = CLIPVisionLoader('CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors')
        model = IPAdapterAdvanced(model, ipadapter, image, 1, 'style transfer precise', 'concat', 0, 1, 'K+mean(V) w/ C penalty', None, mask, clip_vision)

        conditioning2 = CLIPTextEncode('text, logo, worst quality, bad quality, horror, disfigured, nsfw, nude, naked', clip)
        if len(promptlist)>0:
            for prompt in promptlist:
                prompt = prompt+", classical painting"
                conditioning = CLIPTextEncode(prompt, clip)
                latent = EmptyLatentImage(512, 512, 1)
                latent = KSampler(model, 1054924477816377, 35, 5, 'dpmpp_2m', 'karras', conditioning, conditioning2, latent, 1)
                # latent = LatentUpscaleBy(latent, 'nearest-exact', 1.5)
                # latent = KSampler(model, 689292420214014, 35, 5, 'dpmpp_2m', 'karras', conditioning, conditioning2, latent, 0.67)
                image3 = VAEDecode(latent, vae)
                respim = SaveImage(image3)
                finalim=respim['ui']['images'][0]
                image_library.append(finalim)
        else:
            for i in range(10):
                image_library.append(refim)

    return image_library

I modified the load image and save image nodes to accept and return variables, and this function works as expected in jupyter notebook when I use real mode

[pranaykoppula]

I think something about the comfy server starting up prevents the flask app from working

[Chaoses-Ib]

Does connecting to a separate ComfyUI server work, e.g. load('http://127.0.0.1:8188/')?

If it does, you can try to disable the in-process ComfyUI server by:

from comfy_script.runtime.real import *
load(args=ComfyUIArgs('--force-fp16'), no_server=True)
from comfy_script.runtime.real.nodes import *

[Chaoses-Ib]

I modified the load image and save image nodes to accept and return variables, and this function works as expected in jupyter notebook when I use real mode

Virtual mode differs from real mode in several ways. You can't just read ['ui'] and get the images. Instead, do this:

from PIL import Image

image = EmptyImage(512, 512, 1)
image = PreviewImage(image)

image_batch = image.wait()

# Get the first image
image: Image.Image = image_batch[0]

# Get all images in the batch
images: list[Image.Image] = image_batch.wait()

Or with await:

from PIL import Image

image = EmptyImage(512, 512, 1)
image = PreviewImage(image)

image_batch = await image

# Get the first image
image: Image.Image = await image_batch.get(0)

# Get all images in the batch
images: list[Image.Image] = await image_batch

[pranaykoppula]

I was able to make this work in a Jupyter notebook. The issue now seems to be that after the Comfy server starts, I'm not able to send anything to the Flask endpoint.

Even when I set no_server to True, it starts the server

[pranaykoppula]

I just wanna know how to use ComfyScript inside of a flask app

[Chaoses-Ib]

It works fine on my side:

from flask import Flask, request, Response, jsonify

app = Flask(__name__)
from comfy_script.runtime import *
load(args=ComfyUIArgs('--force-fp16'))
from comfy_script.runtime.nodes import *

def comfy_function(prompt:str):
    print('Prompt:', prompt)
    with Workflow():
        image = EmptyImage(512, 512, 1)
        image = PreviewImage(image)

        image_batch = image.wait()
        # Get the first image
        image: Image.Image = image_batch[0]
        # TypeError: Object of type PngImageFile is not JSON serializable
        # return image
        return 'abcd'

@app.route("/generate",methods=['POST'])
def genimages():
    data = request.get_json()
    usr_prompt=data.get('prompt')
    res=comfy_function(usr_prompt)
    response = jsonify(res)
    return response

if __name__ == "__main__":
    app.run(host ='127.0.0.1', port= 11007)

[pranaykoppula]

It's just crickets over here for me.

Very kind you to test it yourself. Thank you.

I'll keep at it.

[Chaoses-Ib]

What's your ComfyUI and ComfyScript version? I tested with the newest version, ComfyUI https://github.com/comfyanonymous/ComfyUI/commit/48eb1399c02bdae7e14b2208c448b69b382d0090 and ComfyScript 167cbd231f9e4b4477fe207d169c4502fba3aca7. The Flask version is 3.0.3 and the Python version is 3.10.10. On Windows 11.

Custom nodes may also cause issues. You can temply disable them by:

load(args=ComfyUIArgs('--disable-all-custom-nodes', '--force-fp16'))

[pranaykoppula]

I installed ComfyUI on a new system yesterday, and installed ComfyScript right after. I'm using Python 3.10 too and I'm using Ubuntu. Unfortunately I do need the custom nodes.

My Flask code looks pretty much identical to yours but I suspect I've made a monumentally stupid mistake somewhere and it probably has nothing to do with ComfyScript.

Thank you for going so far out of your way to help me. I'll edit this post if I'm able to fix it.

[pranaykoppula]

Okay, I honestly don't know what I did right, but I made it work.

Thank you, @Chaoses-Ib for making arguably the most useful ComfyUI extension and for being so helpful.