Closed Godofnothing closed 11 months ago
Hi, thank you for your efforts in enhancing this repository!
TL;DR - We will get back to you after reviewing the functionality, generation score, GPU memory usage, and inference time. Appreciate your patience.
In addition, the base training script src/kd_train_text_to_image.py logs only the total loss to W&B and one may be interested in each particular contribution. I added image logging to W&B as well.
It looks great and seems working properly. We'll verify its functionality and reach out to you.
The original
src/generate.py
generates images one-by-one which leads to under utilization of GPU and as consequence, generation of 30k images takes a while. I've added batched generation of images to speed-up it.
We’d like to clarify that our src/generate.py
with per-image inference was provided to reproduce the generation scores reported in our paper.
We need to examine how batch inference might influence these scores, particularly considering its potential effects on the sampling of random latent codes. Furthermore, the impact of batch size on GPU memory consumption and inference time will also be checked. This evaluation will take some time and thus might cause the delay in our response. We appreciate your kind understanding.
@bokyeong1015 Thanks for your response. Surely, reproducibility is crucial. I hope you'll find these modifications beneficial.
Hi, thanks again for your PR and sorry for the late response. We’ve carefully reviewed the modified code and would like to share our progress.
Before proceeding, would you kindly create separate PRs with two distinct branches? If this is OK for you, please let us know. We'll then set up the corresponding issues and branches and get back to you.
src/kd_train_text_to_image.py
)Thank you for adding this functionality; it works well. One point we'd like to revise is logging {train_loss_sd, train_loss_kd_output, train_loss_kd_feat} instead of {loss_sd, loss_kd_output, loss_kd_feat}. This change would align with our custom CSV logging and enable considering gradient_accumulation_steps and distributed processes. If you create a separate PR, we'll reiterate this comment during the code review phase.
src/generate.py
, src/generate_single_prompt.py
, src/utils/inference_pipeline.py
)We appreciate this valuable feature and would like to incorporate it with a default batch size of 1. Though the inference batch size doesn't influence the reproducibility of generation scores (please see below), minimizing GPU memory usage would be beneficial for users with lower-spec machines. We'll provide a detailed explanation of this feature in our documentation and will appropriately acknowledge your contribution.
Eval Batch Size | 1 | 4 | 8 |
---|---|---|---|
GPU Memory | 4978MiB (4.9GB) | 11590MiB (11.3GB) | 20034MiB (19.6GB) |
Generation Time | 34130s (9.5h) | 27564s (7.7h) | 26313s (7.3h) |
FID | 16.98 | 17.16 | 16.97 |
IS | 31.68 | 31.62 | 31.22 |
CLIP Score | 0.2677 | 0.2677 | 0.2675 |
@bokyeong1015 glad to hear you found these modifications helpful. Yeah I agree, that two pull requests for two separate features are better for maintaining the structure of the project, and I'll take into account your comments and points in the new pull requests.
Thanks for your kind reply! Could you please create two separate PRs for the branches 37-improved-wandb-logger
and 38-batched-image-generation
?
Either of the two options below is fine, so please decide at your convenience. Thank you for your time and help.
Hi, thanks for your work. The results are quite impressive. Below I added a few modifications that can be helpful for experimentation.
The original
src/generate.py
generates images one-by-one which leads to under utilization of GPU and as consequence, generation of 30k images takes a while. I've added batched generation of images to speed-up it.In addition, the base training script
src/kd_train_text_to_image.py
logs only the total loss to W&B and one may be interested in each particular contribution. I added image logging to W&B as well.