nullquant / ComfyUI-BrushNet

ComfyUI BrushNet nodes
Apache License 2.0
571 stars 20 forks source link

Hello author, could you please test it yourself when you have free time? When reading a batch of video frames, if it exceeds 30 frames, the memory will be insufficient. Whether it is an inpaint or an outpaint, please test it yourself and fix it #100

Closed meetedlike closed 3 months ago

meetedlike commented 4 months ago

Hello author, could you please test it yourself when you have free time? When reading a batch of video frames, if it exceeds 30 frames, the memory will be insufficient. Whether it is an inpaint or an outlier, please test it yourself and fix it

brushnet1 brushnet2 brushnet3
nullquant commented 4 months ago

This is workflow with 50 images 512x512 and it takes about 17 Gb of VRAM at maximum:

Screenshot from 2024-05-19 11-48-33

But it depends, of course, on many factors including other models, such as SAM and DINO, which you can do in separate workflow, but most of all on image resolution. In any case you can do it in several batches of less images.

meetedlike commented 4 months ago

Compared with the workflow you sent, I roughly understand why my 30 images got OOM. The reason is that the video frames I uploaded have a resolution of 1080 1920, while you used 512 512. Thank you for your reply. Your attitude towards the problem has made me feel that 1. You really want to do a good job with this plugin. 2. You also feel that you are willing to handle and solve the problem. Thank you. I will test it again when I have time, and also hope that you can optimize it when you have time. I still hope to make an outward extension that can handle one minute of video. If I find any problems, I will submit them. Thank you

nullquant commented 4 months ago

@meetedlike I have found a solution for you. You can use Evolved Sampling from AnimateDiff-Evolved:

Screenshot from 2024-05-23 22-02-59

In Context Options set context_length to number of images which can be loaded into VRAM. Images will be processed in chunks of this size.