search

bmaltais / kohya_ss

Apache License 2.0
9.64k stars 1.24k forks source link

Why kohya_ss faster compare to sd-scripts repository? #1686

Closed LeDuySon closed 9 months ago

LeDuySon commented 12 months ago

Hi everyone, i have tried to run both the kohya_ss webui and script in sd-scripts repo with below setting: "sdxl": false, "v2": false, "v_parameterization": false, "num_cpu_threads_per_process": 4, "pretrained_model_name_or_path": "deliberate_v3.safetensors", "color_aug": false, "flip_aug": false, "random_crop": false, "resolution": "512,512", "cache_latents": true, "cache_latents_to_disk": false, "enable_bucket": true, "min_bucket_reso": 256, "max_bucket_reso": 2048, "bucket_reso_steps": 64, "bucket_no_upscale": true, "reg_data_dir": "", "dataset_repeats": 20, "save_precision": "bf16", "save_every_n_epochs": 1, "save_last_n_epochs": 1, "save_state": false, "train_batch_size": 10, "max_token_length": 150, "xformers": true, "vae": "", "max_train_epochs": 7, "seed": 1, "gradient_checkpointing": true, "gradient_accumulation_steps": 1, "mixed_precision": "bf16", "clip_skip": 1, "noise_offset": 0, "multires_noise_iterations": 0, "multires_noise_discount": 0, "adaptive_noise_scale": 0, "min_timestep": 0, "max_timestep": 1000, "prior_loss_weight": 1.0, "optimizer_type": "AdamW8bit", "learning_rate": 0.0001, "lr_scheduler": "cosine", "lr_warmup_steps": 10, "min_snr_gamma": 0, "scale_v_pred_loss_like_noise_pred": false, "v_pred_like_loss": 0, "weighted_captions": false, "save_model_as": "safetensors", "unet_lr": 0.0001, "text_encoder_lr": 5e-05, "network_module": "networks.lora", "network_dim": 256, "network_alpha": 256, "network_dropout": 0, "network_train_unet_only": false, "network_train_text_encoder_only": false, "dim_from_weights": false, "scale_weight_norms": 0, "no_half_vae": false I have also set accelerate config to be the same as config_files/accelerate/default_config.yaml. I want to ask why the kohya-ss is much faster compare to sd-scripts repository? It took 5 minutes in kohya-ss and ~10 minutes in sd-scripts.

bmaltais commented 12 months ago

Wow... this is a significant difference. The only reason I can see is that perhaps my requirements file is slightly different... of perhaps you tested with the version that does not include the latest sd-scripts updates. I will publish a new release in a few minutes that will include the latest code... Test this new version and see if it is still as fast... curious to know.