[Feature Request] ComfyAPI Progress Status Messages for Custom Frontends

[futureengine-io]

Problem Description ComfyAPI is the best way to create API-based workflows for SD. In creating more complex workflows that take time for inference, keeping the user updated on status is becoming vital.

Proposed Feature Allow the API to report the active node in a running workflow via messages / websockets (socket.io) / to allow the front-end to create a responsive progress indicator (e.g. Now Splitting Frames, Now Generating, Now Swapping Face...)

[sviccc]

Im voting for this up. Please create API endpoint for example /progress, that will return atleast overall progress status in percentage. this is the only thing needed for my mobile phone simplistic frontend ui. Thank you ;-)

[Chaoses-Ib]

The /ws API already supported this. It's just not documented:

{'type': 'status', 'data': {'status': {'exec_info': {'queue_remaining': 0}}, 'sid': 'adc24049-b013-4a58-956b-edbc591dc6e2'}}
{'type': 'status', 'data': {'status': {'exec_info': {'queue_remaining': 1}}}}
{'type': 'execution_start', 'data': {'prompt_id': '3328f0c8-9368-4070-90e7-087e854fe315'}}
{'type': 'execution_cached', 'data': {'nodes': ['9', '15', '5', '8', '12', '13', '16', '6', '1', '14', '0', '2', '20', '11', '17', '7', '10', '19', '3', '4', '18'], 'prompt_id': '3328f0c8-9368-4070-90e7-087e854fe315'}}
{'type': 'executing', 'data': {'node': '21', 'prompt_id': '3328f0c8-9368-4070-90e7-087e854fe315'}}
{'type': 'progress', 'data': {'value': 1, 'max': 15}}
...
{'type': 'progress', 'data': {'value': 15, 'max': 15}}
{'type': 'executing', 'data': {'node': '22', 'prompt_id': '3328f0c8-9368-4070-90e7-087e854fe315'}}
{'type': 'executing', 'data': {'node': '23', 'prompt_id': '3328f0c8-9368-4070-90e7-087e854fe315'}}
{'type': 'executed', 'data': {'node': '23', 'output': {'images': [{'filename': 'C_00001_.png', 'subfolder': '', 'type': 'output'}]}, 'prompt_id': '3328f0c8-9368-4070-90e7-087e854fe315'}}
{'type': 'status', 'data': {'status': {'exec_info': {'queue_remaining': 0}}}}
{'type': 'executing', 'data': {'node': None, 'prompt_id': '3328f0c8-9368-4070-90e7-087e854fe315'}}

The progress message doesn't include the node id (#2425), but you can get it from the previous executing message.

Here is the code ComfyScript used to handle it: https://github.com/Chaoses-Ib/ComfyScript/blob/6ddd8215d24aeb7c2ddbb5a8bcf087fce2f89dec/script/runtime/__init__.py#L78-L158

[seghier]

how to open /ws in the browser? i try: http://127.0.0.1:8188/ws but i get this:

No WebSocket UPGRADE hdr: None
 Can "Upgrade" only to "WebSocket".

[johnr14]

@seghier

From : websockets_api_example.py

Here is my hacked code. I get node NUMBER progress in total workflow, but no information on % done for a certain node.

Time is for last node. This is as good as I could get it. You get STEP number on total STEPS for nodes that have them (or batch ?).

Also, if I open an other ws// client, I will not get progess, only queue information. Why ?

Would like to have 10% increment of node work progress.

2024-08-16 17:44:13 INFO     Queue remaining: 1
2024-08-16 17:44:45 INFO     Progress is 1/4time 0.54 min
2024-08-16 17:45:49 INFO     Progress is 2/4time 1.59 min
2024-08-16 17:46:52 INFO     Progress is 3/4time 2.65 min
2024-08-16 17:47:56 INFO     Progress is 4/4time 3.72 min
2024-08-16 17:47:58 INFO     Queue remaining: 0

def get_images(ws, prompt):
    prompt_id = queue_prompt(prompt)['prompt_id']
    output_images = {}
    progress_time = time.time()
    while True:
        out = ws.recv()
        if isinstance(out, str):
            message = json.loads(out)
            if message['type'] == 'executing':
                data = message['data']
                if data['node'] is None and data['prompt_id'] == prompt_id:
                    break #Execution is done
            elif message['type'] == 'status':
                data = message['data']
                queue_remaining = data['status']['exec_info']['queue_remaining']
                logger.info("Queue remaining: " + str(queue_remaining))
            elif message['type'] == 'execution_start':
                executing = True
                #logger.debug("Executing!")
            elif message['type'] == 'executed':
                data = message['data']
                prompt_id = data['prompt_id']
                logger.debug("Executed : " + prompt_id)
                #task: Task = self._tasks.get(prompt_id)

            elif message['type'] == 'progress':
                data = message['data']
                progress_time_used = str(round((time.time() - start_time) / 60, 2))
                logger.info("Progress is " + str(data['value']) + "/" + str(data['max']) + "time " + progress_time_used + " min")
                progress_time = time.time()
        else:
            continue #previews are binary data

    history = get_history(prompt_id)[prompt_id]
    for o in history['outputs']:
        for node_id in history['outputs']:
            node_output = history['outputs'][node_id]
            if 'images' in node_output:
                images_output = []
                for image in node_output['images']:
                    image_data = get_image(image['filename'], image['subfolder'], image['type'])
                    images_output.append(image_data)
            output_images[node_id] = images_output

    return output_images

I use a logger, but could be put back to printf and update the source file in git with more documentation ?

How do sites that run workflows on cloud get finer % progression ?

Thanks for the great work !

[maxhille]

@johnr14 people on Reddit discussed this issue too and someone had the solution (https://www.reddit.com/r/comfyui/comments/1dpi7ml/not_all_event_types_coming_through_when_connected/)

I now just don't provide a client_id query param at all on the initial WS connect and just use the returned sid as a client_id in all subsequent HTTP requests.