Error when I try to train a model with Kohya

[corvo4791]

prepare optimizer, data loader etc.

===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues For effortless bug reporting copy-paste your error into this form: https://docs.google.com/forms/d/e/1FAIpQLScPB8emS3Thkp66nvqwmjTEgxp8Y9ufuWTzFyr9kJ5AoI47dQ/viewform?usp=sf_link

CUDA SETUP: Loading binary C:\code\kohya\kohya_ss\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda116.dll... use 8-bit AdamW optimizer | {} ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ C:\code\kohya\kohya_ss\train_db.py:480 in │ │ │ │ 477 │ args = parser.parse_args() │ │ 478 │ args = train_util.read_config_from_file(args, parser) │ │ 479 │ │ │ ❱ 480 │ train(args) │ │ 481 │ │ │ │ C:\code\kohya\kohya_ss\train_db.py:163 in train │ │ │ │ 160 │ else: │ │ 161 │ │ trainableparams = unet.parameters() │ │ 162 │ │ │ ❱ 163 │ , _, optimizer = train_util.get_optimizer(args, trainable_params) │ │ 164 │ │ │ 165 │ # dataloaderを準備する │ │ 166 │ # DataLoaderのプロセス数：0はメインプロセスになる │ │ │ │ C:\code\kohya\kohya_ss\library\train_util.py:3249 in get_optimizer │ │ │ │ 3246 │ │ │ │ │ "No PagedLion8bit. The version of bitsandbytes installed seems to be │ │ 3247 │ │ │ │ ) │ │ 3248 │ │ │ │ ❱ 3249 │ │ optimizer = optimizer_class(trainable_params, lr=lr, **optimizer_kwargs) │ │ 3250 │ │ │ 3251 │ elif optimizer_type == "SGDNesterov".lower(): │ │ 3252 │ │ print(f"use SGD with Nesterov optimizer | {optimizer_kwargs}") │ │ │ │ C:\code\kohya\kohya_ss\venv\lib\site-packages\bitsandbytes\optim\adamw.py:52 in init │ │ │ │ 49 │ │ percentile_clipping=100, │ │ 50 │ │ block_wise=True, │ │ 51 │ ): │ │ ❱ 52 │ │ super(AdamW8bit, self).init( │ │ 53 │ │ │ "adam", │ │ 54 │ │ │ params, │ │ 55 │ │ │ lr, │ │ │ │ C:\code\kohya\kohya_ss\venv\lib\site-packages\bitsandbytes\optim\optimizer.py:330 in init │ │ │ │ 327 │ │ │ │ "Invalid weight_decay value: {}".format(weight_decay) │ │ 328 │ │ │ ) │ │ 329 │ │ defaults = dict(lr=lr, betas=betas, eps=eps, weight_decay=weight_decay) │ │ ❱ 330 │ │ super(Optimizer2State, self).init(params, defaults, optim_bits) │ │ 331 │ │ │ │ 332 │ │ if args is None: │ │ 333 │ │ │ args = {} │ │ │ │ C:\code\kohya\kohya_ss\venv\lib\site-packages\bitsandbytes\optim\optimizer.py:96 in init │ │ │ │ 93 │ │ 94 class Optimizer8bit(torch.optim.Optimizer): │ │ 95 │ def init(self, params, defaults, optim_bits=32): │ │ ❱ 96 │ │ super(Optimizer8bit, self).init(params, defaults) │ │ 97 │ │ self.initialized = False │ │ 98 │ │ self.name2qmap = {} │ │ 99 │ │ │ │ C:\code\kohya\kohya_ss\venv\lib\site-packages\torch\optim\optimizer.py:49 in init │ │ │ │ 46 │ │ │ │ 47 │ │ param_groups = list(params) │ │ 48 │ │ if len(param_groups) == 0: │ │ ❱ 49 │ │ │ raise ValueError("optimizer got an empty parameter list") │ │ 50 │ │ if not isinstance(param_groups[0], dict): │ │ 51 │ │ │ param_groups = [{'params': param_groups}] │ │ 52 │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ ValueError: optimizer got an empty parameter list ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ C:\Users\Aldo\AppData\Local\Programs\Python\Python310\lib\runpy.py:196 in _run_module_as_main │ │ │ │ 193 │ main_globals = sys.modules["main"].dict │ │ 194 │ if alter_argv: │ │ 195 │ │ sys.argv[0] = mod_spec.origin │ │ ❱ 196 │ return _run_code(code, main_globals, None, │ │ 197 │ │ │ │ │ "main", mod_spec) │ │ 198 │ │ 199 def run_module(mod_name, init_globals=None, │ │ │ │ C:\Users\Aldo\AppData\Local\Programs\Python\Python310\lib\runpy.py:86 in _run_code │ │ │ │ 83 │ │ │ │ │ loader = loader, │ │ 84 │ │ │ │ │ package = pkg_name, │ │ 85 │ │ │ │ │ spec = mod_spec) │ │ ❱ 86 │ exec(code, run_globals) │ │ 87 │ return run_globals │ │ 88 │ │ 89 def _run_module_code(code, init_globals=None, │ │ │ │ in :7 │ │ │ │ 4 from accelerate.commands.accelerate_cli import main │ │ 5 if name == 'main': │ │ 6 │ sys.argv[0] = re.sub(r'(-script.pyw|.exe)?$', '', sys.argv[0]) │ │ ❱ 7 │ sys.exit(main()) │ │ 8 │ │ │ │ C:\code\kohya\kohya_ss\venv\lib\site-packages\accelerate\commands\accelerate_cli.py:45 in main │ │ │ │ 42 │ │ exit(1) │ │ 43 │ │ │ 44 │ # Run │ │ ❱ 45 │ args.func(args) │ │ 46 │ │ 47 │ │ 48 if name == "main": │ │ │ │ C:\code\kohya\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py:918 in │ │ launch_command │ │ │ │ 915 │ elif defaults is not None and defaults.compute_environment == ComputeEnvironment.AMA │ │ 916 │ │ sagemaker_launcher(defaults, args) │ │ 917 │ else: │ │ ❱ 918 │ │ simple_launcher(args) │ │ 919 │ │ 920 │ │ 921 def main(): │ │ │ │ C:\code\kohya\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py:580 in │ │ simple_launcher │ │ │ │ 577 │ process.wait() │ │ 578 │ if process.returncode != 0: │ │ 579 │ │ if not args.quiet: │ │ ❱ 580 │ │ │ raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd) │ │ 581 │ │ else: │ │ 582 │ │ │ sys.exit(1) │ │ 583 │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ CalledProcessError: Command '['C:\code\kohya\kohya_ss\venv\Scripts\python.exe', './train_db.py', '--enable_bucket', '--min_bucket_reso=256', '--max_bucket_reso=2048', '--pretrained_model_name_or_path=C:/code/stable-diffusion-webui/models/Stable-diffusion/v1-5-pruned.ckpt', '--train_data_dir=C:/code/xKOHYA/images', '--resolution=512,512', '--output_dir=C:/code/xKOHYA/model', '--logging_dir=C:/code/xKOHYA/log', '--save_model_as=safetensors', '--output_name=SNDLG', '--lr_scheduler_num_cycles=1', '--max_data_loader_n_workers=0', '--learning_rate=1e-05', '--lr_scheduler=cosine', '--lr_warmup_steps=76', '--train_batch_size=1', '--max_train_steps=760', '--save_every_n_epochs=10', '--mixed_precision=fp16', '--save_precision=fp16', '--cache_latents', '--optimizer_type=AdamW8bit', '--max_data_loader_n_workers=0', '--bucket_reso_steps=64', '--xformers', '--bucket_no_upscale', '--noise_offset=0.0', '--sample_sampler=euler_a', '--sample_prompts=C:/code/xKOHYA/model\sample\prompt.txt', '--sample_every_n_steps=200']' returned non-zero exit status 1.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

[DarksealStudios]

same issue.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.