search

lllyasviel / ControlNet

Let us control diffusion models!
Apache License 2.0
29.02k stars 2.62k forks source link

Control net and img2img #161

Open UglyStupidHonest opened 1 year ago

UglyStupidHonest commented 1 year ago

Can someone give me any pointers on how to implement image to image together with Control Net in lets say the gradio_canny2image.py , I tried to adapt the img2img original ldm code but Im just not at that level yet :) I would be eternally grateful!

Thank you so much!

haofanwang commented 1 year ago

@UglyStupidHonest Check this repo if you just want to use it in diffusers.

UglyStupidHonest commented 1 year ago

Oh amazing thank you so much !!

un1tz3r0 commented 1 year ago

@UglyStupidHonest and @haofanwang, i've adapted the img2img+controlnet code in that repo, to use on video inputs. my code is here. I use the img2img input to stabilize the output across successive frames by feeding some of the previous output frame back, with motion transferred from the input. many thanks to you @haofanwang, i too tried to adapt the original ldm code and had no luck with it, your repo really saved me alot of frustration

UglyStupidHonest commented 1 year ago

Oh very cool!! Thank you!!

actionless commented 8 months ago

it's already merged: https://github.com/huggingface/diffusers/pull/2407/files#diff-575959cae6329573c802ceca78d9a160b9d25948817a9ff37feb37585f819b05 and works for me

(mb not this particular PR, but latest diffusers got Controlnet+img2img)