voluptuous.error.MultipleInvalid: extra keys not allowed @ data['settings'] why?

[bank010]

2024-06-19 15:08:43 INFO Loading settings from ./outputs/config_lora-20240619-150835.toml... train_util.py:3744 INFO ./outputs/config_lora-20240619-150835 train_util.py:3763 2024-06-19 15:08:43 INFO prepare tokenizer train_util.py:4227 2024-06-19 15:08:54 INFO update token length: 150 train_util.py:4244 INFO Loading dataset config from ./config.toml train_network.py:161 WARNING ignoring the following options because config file is found: train_data_dir, reg_data_dir, in_json / train_network.py:165 設定ファイルが利用されるため以下のオプションは無視されます: train_data_dir, reg_data_dir, in_json
ERROR Invalid user config / ユーザ設定の形式が正しくないようです config_util.py:368 Traceback (most recent call last): File "/data1/SD/kohya_ss/sd-scripts/train_network.py", line 1115, in trainer.train(args) File "/data1/SD/kohya_ss/sd-scripts/train_network.py", line 197, in train blueprint = blueprint_generator.generate(user_config, args, tokenizer=tokenizer) File "/data1/SD/kohya_ss/sd-scripts/library/config_util.py", line 402, in generate sanitized_user_config = self.sanitizer.sanitize_user_config(user_config) File "/data1/SD/kohya_ss/sd-scripts/library/config_util.py", line 365, in sanitize_user_config return self.user_config_validator(user_config) File "/data1/SD/kohya_ss/venv/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 272, in call return self._compiled([], data) File "/data1/SD/kohya_ss/venv/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 595, in validate_dict return base_validate(path, iteritems(data), out) File "/data1/SD/kohya_ss/venv/lib/python3.10/site-packages/voluptuous/schema_builder.py", line 433, in validate_mapping raise er.MultipleInvalid(errors) voluptuous.error.MultipleInvalid: extra keys not allowed @ data['settings'] [2024-06-19 15:08:58,528] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 1765932) of binary: /data1/SD/kohya_ss/venv/bin/python Traceback (most recent call last): File "/data1/SD/kohya_ss/venv/bin/accelerate", line 8, in sys.exit(main()) File "/data1/SD/kohya_ss/venv/lib/python3.10/site-packages/accelerate/commands/accelerate_cli.py", line 47, in main args.func(args) File "/data1/SD/kohya_ss/venv/lib/python3.10/site-packages/accelerate/commands/launch.py", line 1008, in launch_command multi_gpu_launcher(args) File "/data1/SD/kohya_ss/venv/lib/python3.10/site-packages/accelerate/commands/launch.py", line 666, in multi_gpu_launcher distrib_run.run(args) File "/data1/SD/kohya_ss/venv/lib/python3.10/site-packages/torch/distributed/run.py", line 797, in run elastic_launch( File "/data1/SD/kohya_ss/venv/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in call return launch_agent(self._config, self._entrypoint, list(args)) File "/data1/SD/kohya_ss/venv/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 264, in launch_agent raise ChildFailedError( torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

/data1/SD/kohya_ss/sd-scripts/train_network.py FAILED

Failures:

------------------------------------------------------------ Root Cause (first observed failure): [0]: time : 2024-06-19_15:08:58 host : instance-tnwhosh1 rank : 0 (local_rank: 0) exitcode : 1 (pid: 1765932) error_file: traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html ============================================================ 15:08:59-722054 INFO Training has ended.

[kirby-jack]

your config file is causing the issue, can you please post the settings you have in config_lora-20240619-150835.toml