yeates / PromptFix

[NeurIPS 24] PromptFix: You Prompt and We Fix the Photo
Apache License 2.0
626 stars 36 forks source link

Details of High-frequency guidance #8

Open dhryougit opened 1 month ago

dhryougit commented 1 month ago

Thank you for the excellent work! I found the high-frequency sampling part of your paper intriguing and would like to reproduce it, so I’m seeking more details. I would appreciate it if you could answer the following questions:

  1. Is it correct that the LoRA parameters are re-initialized at the start of each sampling and are only trained during the sampling process?
  2. Which optimizer for training LoRA parameters did you use during training while sampling? Could you provide more details?
  3. In the code, it seems like HGS is directly applied to z, but in the paper, it seems that HGS is only used for training the decoder(lora parameter). Which one is correct?