Kosinkadink / ComfyUI-VideoHelperSuite

Nodes related to video workflows
GNU General Public License v3.0
513 stars 90 forks source link

Seeking clarity on what `prompt` should be in combine_video #112

Closed slytechnical closed 8 months ago

slytechnical commented 8 months ago

I used https://github.com/pydn/ComfyUI-to-Python-Extension to export a ComfyUI workflow that includes the VideoCombine node.

The resulting code was

vhs_videocombine_125 = vhs_videocombine.combine_video(
    frame_rate=25,
    loop_count=0,
    filename_prefix="pexelsdance1-",
    format="video/ProRes",
    pingpong=False,
    save_output=True,
    images=get_value_at_index(vaedecode_10, 0),
    unique_id=9718558386888217146,
)

This results in the error

Traceback (most recent call last):
  File "/workspace/ComfyUI/ComfyUI-to-Python-Extension/workflow_api.py", line 423, in <module>
    main()
  File "/workspace/ComfyUI/ComfyUI-to-Python-Extension/workflow_api.py", line 406, in main
    vhs_videocombine_125 = vhs_videocombine.combine_video(
  File "/workspace/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite/videohelpersuite/nodes.py", line 123, in combine_video
    kwargs = prompt[unique_id]['inputs']
TypeError: 'NoneType' object is not subscriptable

It seems here https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite/blob/c8a4fb77e2290b65d118b965d6753482c8b162bf/videohelpersuite/nodes.py#L123C16-L123C16 there is supposed to be a prompt passed in which is some kind of a dictionary with that unique_id as a key and contains an inputs but I have no clue what the value should be of those inputs. Is there anywhere I can look for additional info about what this prompt dictionary should be?

Side node, I tried just adding a blank one like

 prompt_dict = {
    9718558386888217146: {  # Use your unique_id here
        'inputs': {}  # You can populate this with relevant inputs if needed
    }
}

and oddly that made the whole workflow loop...?

AustinMroz commented 8 months ago

You seem to be pretty close in your analysis. prompt contains information on the workflow. It's intended purpose is so this information can be stored in output images and you can recreate the workflow from the image. While prompt is used for it's intended purpose further into the node, the line of interest

kwargs = prompt[unique_id]['inputs']

is a bit of a hack. A system was recently added for video_formats to dynamically add additional inputs to a node. For example, most video formats will have crf as an option for quality, but the animated image options (gif and webp) don't. The unfortunate issue, is that ComfyUI properly sanitizes node inputs so that these extra input options don't actually get passed to the VideoCombine method. In other words, prompt is used to access the unsanitized inputs given for the VideoCombine node. Since the ProRes format doesn't have any of these extra options, you could safely replace the line with

kwargs = {}
slytechnical commented 8 months ago

Thank you so much for the quick and thorough response.

slytechnical commented 8 months ago

BTW if anybody stumbles on this issue later the hack fix for me (after pulling a recent fix and switching to h264) ended up being to just manually put the correct kwargs

kwargs = {'frame_rate': 25, 'loop_count': 0, 'filename_prefix': 'pexelsdance1-', 'format': 'video/h264-mp4', 'pix_fmt': 'yuv420p', 'crf': 19, 'save_metadata': True, 'pingpong': False, 'save_output': True, 'images': ['10', 0]}