Progress indicator endpoint

[eyegeegeewhy]

Feature Idea

Would it be possible to get some form of progress indicator added to the /queue endpoint or a new endpoint. I appreciate that it's in the web socket but it would be nice to be able to get this via http. In my setup I can't use a web socket client so I'm polling /history waiting for images to appear.

Existing Solutions

No response

Other

No response

[RandomGitUser321]

It's already there for when it's ksampling. You get current step and total steps through message['type'] == 'progress': and the entries will be called value and max

In the websocket example you can modify the get_images function to do something like this(this is from a ComfyUI API<>Gradio example I made that includes progress and binary preview decoding):

def get_images(ws, prompt):
    global preview_image, active, step_current, step_total
    prompt_id = queue_prompt(prompt)['prompt_id']
    output_images = {}
    while True:
        out = ws.recv()
        if isinstance(out, str):
            message = json.loads(out)
            if message['type'] == 'executing':
                data = message['data']
                if data['node'] is None and data['prompt_id'] == prompt_id:
                    preview_image = None #clear these globals on completion just in case
                    step_current = None
                    step_total = None
                    active = False
                    break #Execution is done
            elif message['type'] == 'progress':
                data = message['data']
                step_current = data['value']
                step_total = data['max']
        else:
            bytesIO = BytesIO(out[8:])
            preview_image = Image.open(bytesIO) # This is your preview in PIL image format

    history = get_history(prompt_id)[prompt_id]
    for node_id in history['outputs']:
        node_output = history['outputs'][node_id]
        images_output = []
        if 'images' in node_output:
            for image in node_output['images']:
                image_data = get_image(image['filename'], image['subfolder'], image['type'])
                images_output.append(image_data)
        output_images[node_id] = images_output

    return output_images

Here's the full example over in the PRs: https://github.com/comfyanonymous/ComfyUI/pull/5024

[blindmonkey]

In my workflows I make pretty extensive use of batching and rebatching. One thing I'm noticing about the WS endpoint is that it doesn't indicate which batch is being operated on, or how many batches there are. Since I'm creating the workflows through the API, I can figure out the latter, but the lack of "current batch" information makes it difficult to give the user an accurate progress report.

Would it be possible to add this information to the WS endpoint?

FWIW, I also think it'd be useful for the progress to be exposed in the /queue endpoint as well

[RandomGitUser321]

within the @routes.get("/queue") API call in server.py:

        @routes.get("/queue")
        async def get_queue(request):
            queue_info = {}
            current_queue = self.prompt_queue.get_current_queue()
            queue_info['queue_running'] = current_queue[0]
            queue_info['queue_pending'] = current_queue[1]
            return web.json_response(queue_info)

If you add a function to your websocket script app along the lines of the following and something that calls it(you'd need to run it after some delay of initially sending the prompt to give comfy enough time to prepare, and then call it again on some time interval):

def get_queue():
    url = f"http://{server_address}/queue"
    response = requests.get(url)
    return response.json()
testing = get_queue()
print(len(testing["queue_pending"]))

You'll see that you do indeed have the remaining length of the queue. So between this and what I showed in my previous post, you can definitely know both the queue remaining and current image step progress, while using the websocket api.

To test it out, I queued a batch of 10 in comfy and then ran a script with that code (+the couple imports it needed like json, requests and the server address), and in the console, it said 9. Which means that comfy was currently working on 1 and that 9 were remaining after it, for a total queue length of 10.

Here was the code for the simple testing app that only get's the pending queue count:

import json
import requests

server_address = "127.0.0.1:8188"

def get_queue():
    url = f"http://{server_address}/queue"
    response = requests.get(url)
    return response.json()
testing = get_queue()
print(len(testing["queue_pending"]))

[blindmonkey]

That has not been my experience. It's certainly the case if I make multiple requests (i.e. each /prompt adds something to the /queue).

I'm talking about a work flow where your latent chain as follows: Empty Latent -> Rebatch Latents -> KSampler

For the Empty Latent, set the batch_size to 4 or something, and for Rebatch Latents set the batch_size to 1. Then send a single prompt with that workflow.

I believe you'll find that you get a bunch of progress events going from 1 to the number of latent steps, and then you'll get another set of events going from 1. You will get 4 sets of events total, with no information as to which batch is being processed.