search

sangminkim-99 / Sketch-Guided-Text-To-Image

Unofficial implementation of Sketch-Guided Text-to-Image Diffusion Models
8 stars 0 forks source link

Experiment Results #1

Closed WenB777 closed 1 year ago

WenB777 commented 1 year ago

Hello, I have read the code of your SKETCH-GUIDED-TEXT-IMAGE, and I think it is quite good. May I ask what kind of effect you will get?

sangminkim-99 commented 1 year ago

Hi, @WenB777!

Thank you for your interest in my implementation! Currently, I'm working on reproducing the bike example mentioned in the paper. However, I've come across a couple of missing pieces in the paper's documentation. Specifically, I couldn't find the training images and the random seed used for the model.

I'm doing my best to fill in these gaps and ensure my implementation aligns with the paper's intentions. Once I achieve results that are somewhat similar to what's described in the paper, I will upload the experimental outcomes.

Thank you for your understanding and support!

WenB777 commented 1 year ago

Thank you for your reply, I am looking forward to your experimental results, I have a question to ask you, only Sketch loss is used for training in the paper and in your code, without introducing the original diffusion loss, is such a training process complete and reasonable? I am interested in this direction and look forward to your recovery.

sangminkim-99 commented 1 year ago

Hi, @WenB777.

I wanted to clarify that my focus lies in generating rooms rather than objects, which is why reproducing the desired results may take additional time. However, I believe the paper suggests an excellent approach that involves hijacking certain features of the diffusion model to predict edge maps, without the need to retrain the entire model.

While my current results are still a work in progress and not yet fully refined, I wanted to share my intermediate findings with you. I hope these results can be of some assistance to you.

Text Image w/ Text Sketch Image w/ Sketch+Text
A cozy morning room image image (1) image (2)

Thank you!

Sketch Image from Victoria_Regen.

WenB777 commented 1 year ago

Thank you for your detailed explanation. After looking at your sample drawings, the generation effect is quite good. I have an idea, which is modified on your current code and is being implemented recently, and I hope to consult you more about related work

sangminkim-99 commented 1 year ago

Hi @WenB777!

I have decided to close this issue for now. However, if you have more questions or would like to continue discussing the idea further, please feel free to reach out to me via email. I would be more than happy to continue the conversation!

Thank you.