Closed kevinjdolan closed 2 years ago
I was able to disable the progress produced by the guided-diffusion library through a monkey-patched hack:
def _disable_progress():
from guided_diffusion.gaussian_diffusion import GaussianDiffusion
orig_ddim = GaussianDiffusion.ddim_sample_loop_progressive
orig_plms = GaussianDiffusion.plms_sample_loop_progressive
def new_ddim(*args, **kwargs):
kwargs['progress'] = False
return orig_ddim(*args, **kwargs)
def new_plms(*args, **kwargs):
kwargs['progress'] = False
return orig_plms(*args, **kwargs)
GaussianDiffusion.ddim_sample_loop_progressive = new_ddim
GaussianDiffusion.plms_sample_loop_progressive = new_plms
This succeeds at removing the progress bars.
A cleaner way would be to accept an environment variable, perhaps here:
runner.py#373
show_progress = not os.environ.get('DISCOART_DISABLE_PROGRESS')
if args.diffusion_sampling_mode == 'ddim':
samples = sample_fn(
model,
(args.batch_size, 3, side_y, side_x),
clip_denoised=args.clip_denoised,
model_kwargs={},
cond_fn=cond_fn,
progress=show_progress,
skip_timesteps=skip_steps,
init_image=init,
randomize_class=args.randomize_class,
eta=args.eta,
transformation_fn=lambda x: symmetry_transformation_fn(
x, args.use_horizontal_symmetry, args.use_vertical_symmetry
),
transformation_percent=args.transformation_percent,
)
else:
samples = sample_fn(
model,
(args.batch_size, 3, side_y, side_x),
clip_denoised=args.clip_denoised,
model_kwargs={},
cond_fn=cond_fn,
progress=show_progress,
skip_timesteps=skip_steps,
init_image=init,
randomize_class=args.randomize_class,
order=2,
)
Apparently this does not solve my problem, however! I am seeing a lot of newlines being generated but I am not sure where they are coming from, and this is not an issue when I am running it interactively. Odd behavior!
For reference, this is the output in the logging file that I am seeing:
discoart-8bdf8c8e141c11edb869751f290d0983
ââââââââââââââââââââââââââââââŦââââââââââââââââââââââââââââââââââââââââââââââââââŽ
â Argument â Value â
ââââââââââââââââââââââââââââââŧââââââââââââââââââââââââââââââââââââââââââââââââââ¤
â batch_name â None â
â batch_size* â 4 â
â clamp_grad â True â
â clamp_max â 0.05 â
â clip_denoised â False â
â clip_guidance_scale* â 524.288 â
â clip_models â ['ViT-B-32::openai', 'ViT-B-16::openai', â
â â 'RN50::openai'] â
â clip_models_schedules â None â
â cut_ic_pow* â [1.0]*800+[4.0]*200 â
â cut_icgray_p â [0.2]*400+[0]*600 â
â cut_innercut* â [16]*400+[48]*600 â
â cut_overview* â [48]*400+[16]*600 â
â cut_schedules_group â None â
â cutn_batches* â 1 â
â diffusion_model â 512x512_diffusion_uncond_finetune_008100 â
â diffusion_model_config â None â
â diffusion_sampling_mode â ddim â
â display_rate â None â
â eta â 0.8 â
â gif_fps â 20 â
â gif_size_ratio â 0.5 â
â init_image â None â
â init_scale â 1000 â
â n_batches* â 1 â
â name_docarray* â discoart-8bdf8c8e141c11edb869751f290d0983 â
â on_misspelled_token â ignore â
â perlin_init â False â
â perlin_mode â mixed â
â rand_mag â 0.05 â
â randomize_class â True â
â range_scale* â 250 â
â sat_scale â 0 â
â save_rate* â 10 â
â seed* â 3605636296 â
â skip_augs â False â
â skip_event â None â
â skip_steps â 0 â
â steps* â 100 â
â stop_event â None â
â text_clip_on_cpu â False â
â text_prompts* â ['Overheard at the Orchard', 'Sven Nordqvist â
â â Illustration'] â
â transformation_percent â [0.09] â
â truncate_overlength_prompt â False â
â tv_scale â 0 â
â use_horizontal_symmetry â False â
â use_secondary_model â True â
â use_vertical_symmetry â False â
â width_height* â [512, 512] â
â°âââââââââââââââââââââââââââââ´ââââââââââââââââââââââââââââââââââââââââââââââââââ¯
showing all args (bold * args are non-default)
Loading diffusion model 512x512_diffusion_uncond_finetune_008100...
Found model disco-diffusion/512x512_diffusion_uncond_finetune_008100 in local cache!
...done in 0.00s: (Loading diffusion model 512x512_diffusion_uncond_finetune_008100)
Augmenting disco cache path to: /cache/models/disco-diffusion/512x512_diffusion_uncond_finetune_008100
Loading diffusion model secondary...
Found model disco-diffusion/secondary in local cache!
...done in 0.00s: (Loading diffusion model secondary)
Augmenting disco cache path to: /cache/models/disco-diffusion/secondary
2022-08-04 17:40:49,826 - discoart - INFO - preparing models...
Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off]
Loading model from: /usr/local/lib/python3.10/site-packages/lpips/weights/v0.1/vgg.pth
2022-08-04 17:40:51,561 - discoart - INFO - creating artworks `discoart-8bdf8c8e141c11edb869751f290d0983` (0/1)...
2022-08-04 17:43:58,149 - discoart - INFO - done! discoart-8bdf8c8e141c11edb869751f290d0983
Yes it would be great to kill the TQDM/progress bar stuff for integration use cases.
from 0.11.2 set env DISCOART_DISABLE_TQDM=1
btw, we are K8s and cloud-native expert, more expert than diffusion đ , if you encounter any problem when deploying DiscoArt in production env such as K8s, please let us know @kevinjdolan
FYI @jina-ai/engineering
I was trying to search through the code to figure out where the progress bars are being generated, but I could not locate them. I assume they are being powered by TQDM.
I am getting frustrating output when the logs are output to a file in the context of a Docker container running on Kubernetes. I would like to either remove the TQDM rendering altogether, or modify so that it is not producing the input characters intended for a live console.
Is there a documented way to control the TQDM rendering? If not, can you point me in the right direction for how to control these?