Linaqruf / kohya-trainer

Adapted from https://note.com/kohya_ss/n/nbf7ce8d80f29 for easier cloning
Apache License 2.0
1.83k stars 300 forks source link

Bug '--config_file=/content/LoRA/config/config_file.toml']' returned non-zero exit status 1. #253

Closed ApchDio closed 1 year ago

ApchDio commented 1 year ago

Hello!

Loading settings from /content/LoRA/config/config_file.toml... /content/LoRA/config/config_file prepare tokenizer update token length: 225 Load dataset config from /content/LoRA/config/dataset_config.toml prepare images. found directory /content/LoRA/train_data contains 0 image files ignore subset with image_dir='/content/LoRA/train_data': no images found / 画像が見つからないためサブセットを無視します found directory /content/LoRA/train_data/100_Tom contains 19 image files illegal char in file (not UTF-8) / ファイルにUTF-8以外の文字があります: /content/LoRA/train_data/100_Tom/3.txt ╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /content/kohya-trainer/train_network.py:752 in │ │ │ │ 749 │ args = parser.parse_args() │ │ 750 │ args = train_util.read_config_from_file(args, parser) │ │ 751 │ │ │ ❱ 752 │ train(args) │ │ 753 │ │ │ │ /content/kohya-trainer/train_network.py:117 in train │ │ │ │ 114 │ │ │ } │ │ 115 │ │ │ 116 │ blueprint = blueprint_generator.generate(user_config, args, tokeni │ │ ❱ 117 │ train_dataset_group = config_util.generate_dataset_group_by_bluepr │ │ 118 │ │ │ 119 │ current_epoch = Value("i", 0) │ │ 120 │ current_step = Value("i", 0) │ │ │ │ /content/kohya-trainer/library/config_util.py:375 in │ │ generate_dataset_group_by_blueprint │ │ │ │ 372 │ dataset_klass = FineTuningDataset │ │ 373 │ │ │ 374 │ subsets = [subset_klass(asdict(subset_blueprint.params)) for sub │ │ ❱ 375 │ dataset = dataset_klass(subsets=subsets, asdict(dataset_blueprin │ │ 376 │ datasets.append(dataset) │ │ 377 │ │ 378 # print info │ │ │ │ /content/kohya-trainer/library/train_util.py:1068 in init │ │ │ │ 1065 │ │ │ │ ) │ │ 1066 │ │ │ │ continue │ │ 1067 │ │ │ │ │ ❱ 1068 │ │ │ img_paths, captions = load_dreambooth_dir(subset) │ │ 1069 │ │ │ if len(img_paths) < 1: │ │ 1070 │ │ │ │ print(f"ignore subset with image_dir='{subset.image_d │ │ 1071 │ │ │ │ continue │ │ │ │ /content/kohya-trainer/library/train_util.py:1040 in load_dreambooth_dir │ │ │ │ 1037 │ │ │ # 画像ファイルごとにプロンプトを読み込み、もしあればそち │ │ 1038 │ │ │ captions = [] │ │ 1039 │ │ │ for img_path in img_paths: │ │ ❱ 1040 │ │ │ │ cap_for_img = read_caption(img_path, subset.caption_e │ │ 1041 │ │ │ │ if cap_for_img is None and subset.class_tokens is Non │ │ 1042 │ │ │ │ │ print(f"neither caption file nor class tokens are │ │ 1043 │ │ │ │ │ captions.append("") │ │ │ │ /content/kohya-trainer/library/train_util.py:1023 in read_caption │ │ │ │ 1020 │ │ │ │ │ │ │ lines = f.readlines() │ │ 1021 │ │ │ │ │ │ except UnicodeDecodeError as e: │ │ 1022 │ │ │ │ │ │ │ print(f"illegal char in file (not UTF-8) │ │ ❱ 1023 │ │ │ │ │ │ │ raise e │ │ 1024 │ │ │ │ │ │ assert len(lines) > 0, f"caption file is empt │ │ 1025 │ │ │ │ │ │ caption = lines[0].strip() │ │ 1026 │ │ │ │ │ break │ │ │ │ /content/kohya-trainer/library/train_util.py:1020 in read_caption │ │ │ │ 1017 │ │ │ │ if os.path.isfile(cap_path): │ │ 1018 │ │ │ │ │ with open(cap_path, "rt", encoding="utf-8") as f: │ │ 1019 │ │ │ │ │ │ try: │ │ ❱ 1020 │ │ │ │ │ │ │ lines = f.readlines() │ │ 1021 │ │ │ │ │ │ except UnicodeDecodeError as e: │ │ 1022 │ │ │ │ │ │ │ print(f"illegal char in file (not UTF-8) │ │ 1023 │ │ │ │ │ │ │ raise e │ │ │ │ /usr/lib/python3.10/codecs.py:322 in decode │ │ │ │ 319 │ def decode(self, input, final=False): │ │ 320 │ │ # decode input (taking the buffer into account) │ │ 321 │ │ data = self.buffer + input │ │ ❱ 322 │ │ (result, consumed) = self._buffer_decode(data, self.errors, f │ │ 323 │ │ # keep undecoded input until the next call │ │ 324 │ │ self.buffer = data[consumed:] │ │ 325 │ │ return result │ ╰──────────────────────────────────────────────────────────────────────────────╯ UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte ╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /usr/local/bin/accelerate:8 in │ │ │ │ 5 from accelerate.commands.accelerate_cli import main │ │ 6 if name == 'main': │ │ 7 │ sys.argv[0] = re.sub(r'(-script.pyw|.exe)?$', '', sys.argv[0]) │ │ ❱ 8 │ sys.exit(main()) │ │ 9 │ │ │ │ /usr/local/lib/python3.10/dist-packages/accelerate/commands/accelerate_cli.p │ │ y:45 in main │ │ │ │ 42 │ │ exit(1) │ │ 43 │ │ │ 44 │ # Run │ │ ❱ 45 │ args.func(args) │ │ 46 │ │ 47 │ │ 48 if name == "main": │ │ │ │ /usr/local/lib/python3.10/dist-packages/accelerate/commands/launch.py:1104 │ │ in launch_command │ │ │ │ 1101 │ elif defaults is not None and defaults.compute_environment == Com │ │ 1102 │ │ sagemaker_launcher(defaults, args) │ │ 1103 │ else: │ │ ❱ 1104 │ │ simple_launcher(args) │ │ 1105 │ │ 1106 │ │ 1107 def main(): │ │ │ │ /usr/local/lib/python3.10/dist-packages/accelerate/commands/launch.py:567 in │ │ simple_launcher │ │ │ │ 564 │ process = subprocess.Popen(cmd, env=current_env) │ │ 565 │ process.wait() │ │ 566 │ if process.returncode != 0: │ │ ❱ 567 │ │ raise subprocess.CalledProcessError(returncode=process.return │ │ 568 │ │ 569 │ │ 570 def multi_gpu_launcher(args): │ ╰──────────────────────────────────────────────────────────────────────────────╯ CalledProcessError: Command '['/usr/bin/python3', 'train_network.py', '--sample_prompts=/content/LoRA/config/sample_prompt.txt', '--dataset_config=/content/LoRA/config/dataset_config.toml', '--config_file=/content/LoRA/config/config_file.toml']' returned non-zero exit status 1.

ApchDio commented 1 year ago

Sorry, my problem with text file