open-mmlab / FoleyCrafter

FoleyCrafter: Bring Silent Videos to Life with Lifelike and Synchronized Sounds. AI拟音大师,给你的无声视频添加生动而且同步的音效 😝
https://foleycrafter.github.io/
Apache License 2.0
423 stars 38 forks source link

waiting a comfyui custom node #9

Open whmc76 opened 2 months ago

doogyhatts commented 2 months ago

So far I couldn't find anyone that is willing to create the custom node for this project, which is very weird. I am using the command-line to get this project to work at the moment.

There is a 5 min loading duration and a 12 min inference duration for each video. Have to batch everything together. A custom node will save about 5 min each time we change the seed number as the models have been loaded. The inference duration could be reduced using a GPU with 16gb vram. Mine is only 8gb.

And obviously, having a custom node will save a lot of disk space, since you don't need to setup a specific conda environment for it. I found a tutorial on writing a custom node, so I will try to make one myself.

whmc76 commented 1 month ago

So far I couldn't find anyone that is willing to create the custom node for this project, which is very weird. I am using the command-line to get this project to work at the moment.

There is a 5 min loading duration and a 12 min inference duration for each video. Have to batch everything together. A custom node will save about 5 min each time we change the seed number as the models have been loaded. The inference duration could be reduced using a GPU with 16gb vram. Mine is only 8gb.

And obviously, having a custom node will save a lot of disk space, since you don't need to setup a specific conda environment for it. I found a tutorial on writing a custom node, so I will try to make one myself.

This is really strange, I think it's even necessary in all video flows, looking forward to your project, even the simplest implementation is fine

doogyhatts commented 1 month ago

There is one right now by Smthemex. https://github.com/smthemex/ComfyUI_FoleyCrafter

Seems to be slower than the command-line.