search

exx8 / differential-diffusion

336 stars 17 forks source link

Using sdxl base model only to generate results #13

Open zengjie617789 opened 4 months ago

zengjie617789 commented 4 months ago

I only use sdxl base model to run sdxl/run.py, but got the result below: ![Uploading image.png…]()

anyone who can help will be appreciated.

exx8 commented 4 months ago

I am not able to open the picture. For using only the base model (without the refiner), remove refiner pipe, which begins here: https://github.com/exx8/differential-diffusion/blob/d78f3f3f5181d319c8a96234393d545cec7eba11/SDXL/run.py#L55C1-L59C80 In this line: https://github.com/exx8/differential-diffusion/blob/d78f3f3f5181d319c8a96234393d545cec7eba11/SDXL/run.py#L53C45-L53C86 remove , denoising_end=0.8, output_type="latent" at the end you will also need to switch .images to .images[0]

zengjie617789 commented 4 months ago

@exx8 thanks for the instant answer. I test the work to the face inpainting. Here are results below: original img: 960x602_wmkn_313154488528 mask img: depth_map_head results: image (2)-dd image-dd

prompt:a chinese man spread hands and yelling, in a red t-shirt I think it can as a post process after swapping face or related work such as InstantID.

zengjie617789 commented 4 months ago

I test it with the default seetings, but results I got are more like anime style sometimes. I dont know why. image

Furthermore, anyone who can explain why it set guidance_scale so high as well as steps?

exx8 commented 3 months ago

I actually tested diff diff with no guidance in one of the experiments and it works well. To be honest, I'm not an AI artisan, so the high gs and steps might be redundant. Try using lower gs, it might solve the issue.