Open runner22k opened 1 year ago
The extension is pretty simple. It extracts the faces of the video and align them (so it even works great with upside down faces). It put the extracted faces in batches of batch_size to be processed at once, in a similar way to img2img. You can use controlnet, simply by activating as you normally would, but if you do, you need to set batch_size to 1. Controlnet changed something since I've created this extension, and now controlnet doesn't handle batches anymore. If you have issues making it work, tell me, I'll probably make a quick tutorial video on youtube.
The extension is pretty simple. It extracts the faces of the video and align them (so it even works great with upside down faces). It put the extracted faces in batches of batch_size to be processed at once, in a similar way to img2img. You can use controlnet, simply by activating as you normally would, but if you do, you need to set batch_size to 1. Controlnet changed something since I've created this extension, and now controlnet doesn't handle batches anymore. If you have issues making it work, tell me, I'll probably make a quick tutorial video on youtube.
Oh! thanks for the reply. I will change batch size and check. Lets hope. A tutorial is always welcome. :) Is lora supported too?
And one more question, is it possible for you to add an option to swap the face and then apply user selected art style? May be we need to pass the image back to controlnet and generate. which might double the time. But, Your extension's feature to regenerate frames will still keep the entire process at roughly around 50% than sd-cn animation. Only thing missing is applying art style to the entire video.
is it possible to save setting to a txt file, so that it will be good to reuse same setting again if need.
something is going wrong, I can see a randomly bouncing rounded square when I apply controlnet. please upload a tutorial if possible. Thanks in advance.
I was able to get the extension up and running after some tries, as it needed a package to be compiled and user should have Windows SDK too.
Now I am trying to test it. but I don't see any effect on the video. even art style specified in the prompt is not applied. Dear developer, is there any documentation or tutorial available? How to use controlnet with this extension?