search

JunzheJosephZhu / HiFA

Apache License 2.0
185 stars 5 forks source link

Results of multi-step denoise and single-step denoise #12

Open OrangeSodahub opened 2 months ago

OrangeSodahub commented 2 months ago

Hi, I notice that in your code you have both options of multi-steps denoise and single-step denoise. I wonder that the results showed in your paper belongs to which one? If it is single-step denoise, so how was the results of multi-steps denoise? Did multi-step denoise better than single-step denoise?

Sorry after many tries I still cannot run your code in my machine so didn't validate myself. Thanks!

HiFA-team commented 2 months ago

I think for some prompts it was 4 step, for some it was single-step. Doing multistep denoising between 1-6 steps didn't cause a huge difference. Increasing the step number to above 10 caused a huge difference, the generated object were more color shifted and shrinked, and even more so if I use a DPM solver.

This paper is also implemented on threestudio.

Regarding not being able to run the code, maybe check this out: https://github.com/JunzheJosephZhu/HiFA/issues/9#issuecomment-2056222628

On Sat, May 4, 2024 at 10:22 PM YangXiuyu @.***> wrote:

Hi, I notice that in your code you have both options of multi-steps denoise and single-step denoise. I wonder that the results showed in your paper belongs to which one? If it is single-step denoise, so how was the results of multi-steps denoise? Did multi-step denoise better than single-step denoise?

Sorry after many tries I still cannot run your code in my machine so didn't validate myself. Thanks!

— Reply to this email directly, view it on GitHub https://github.com/JunzheJosephZhu/HiFA/issues/12, or unsubscribe https://github.com/notifications/unsubscribe-auth/BAEVSEGMXQJXGPKTRFK43SLZATVJJAVCNFSM6AAAAABHG327JSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGI3TSMBRHA3DQNI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

OrangeSodahub commented 2 months ago

@HiFA-team Hi, thanks for your information. Do you have any examples showing 'color-shifting' or 'shrink', thanks!

HiFA-team commented 2 months ago

Nope, single ddim denoising step just doesnt work. There are two problems here: 1. DDIM typically only works with >10 steps. 2. CFG=100 will make this worse, although the pseudo GT is much more clear when num_steps>20. I feel like having many more steps just cancels the effect caused by CFG=100(I like to think of DDIM as an SDE solver, and CFG to be some sort of error/noise that happened in the solving process when the integrator's step size is too big)

On Wed, May 8, 2024 at 10:09 PM YangXiuyu @.***> wrote:

@HiFA-team https://github.com/HiFA-team Hi, thanks for your information. Do you have any examples showing 'color-shifting' or 'shrink', thanks!

— Reply to this email directly, view it on GitHub https://github.com/JunzheJosephZhu/HiFA/issues/12#issuecomment-2100676293, or unsubscribe https://github.com/notifications/unsubscribe-auth/BAEVSEFPFDHIUWCBHZASBK3ZBIWZXAVCNFSM6AAAAABHG327JSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBQGY3TMMRZGM . You are receiving this because you were mentioned.Message ID: @.***>