bmaltais / kohya_ss

Apache License 2.0
9.54k stars 1.23k forks source link

UnicodeDecodeError: 'gbk' codec can't decode byte 0x80 in position 64: illegal multibyte sequence #539

Closed mlsjla closed 8 months ago

mlsjla commented 1 year ago

Traceback (most recent call last): File "D:\code\test\kohya_ss\venv\lib\site-packages\transformers\modeling_utils.py", line 415, in load_state_dict return torch.load(checkpoint_file, map_location="cpu") File "D:\code\test\kohya_ss\venv\lib\site-packages\torch\serialization.py", line 705, in load with _open_zipfile_reader(opened_file) as opened_zipfile: File "D:\code\test\kohya_ss\venv\lib\site-packages\torch\serialization.py", line 242, in init super(_open_zipfile_reader, self).init(torch._C.PyTorchFileReader(name_or_buffer)) RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "D:\code\test\kohya_ss\venv\lib\site-packages\transformers\modeling_utils.py", line 419, in load_state_dict if f.read(7) == "version": UnicodeDecodeError: 'gbk' codec can't decode byte 0x80 in position 64: illegal multibyte sequence

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "D:\code\test\kohya_ss\train_network.py", line 724, in train(args) File "D:\code\test\kohya_ss\train_network.py", line 135, in train textencoder, vae, unet, = train_util.load_target_model( File "D:\code\test\kohya_ss\library\train_util.py", line 2649, in load_target_model text_encoder, vae, unet = model_util.load_models_from_stable_diffusion_checkpoint(args.v2, name_or_path, device) File "D:\code\test\kohya_ss\library\model_util.py", line 898, in load_models_from_stable_diffusion_checkpoint text_model = CLIPTextModel.from_pretrained("openai/clip-vit-large-patch14").to(device) File "D:\code\test\kohya_ss\venv\lib\site-packages\transformers\modeling_utils.py", line 2301, in from_pretrained state_dict = load_state_dict(resolved_archive_file) File "D:\code\test\kohya_ss\venv\lib\site-packages\transformers\modeling_utils.py", line 431, in load_state_dict raise OSError( OSError: Unable to load weights from pytorch checkpoint file for 'C:\Users\MJ/.cache\huggingface\hub\models--openai--clip-vit-large-patch14\snapshots\8d052a0f05efbaefbc9e8786ba291cfdf93e5bff\pytorch_model.bin' at 'C:\Users\MJ/.cache\huggingface\hub\models--openai--clip-vit-large-patch14\snapshots\8d052a0f05efbaefbc9e8786ba291cfdf93e5bff\pytorch_model.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True. Traceback (most recent call last): File "C:\Users\MJ\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\MJ\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "D:\code\test\kohya_ss\venv\Scripts\accelerate.exe__main__.py", line 7, in File "D:\code\test\kohya_ss\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 45, in main args.func(args) File "D:\code\test\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 1104, in launch_command simple_launcher(args) File "D:\code\test\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 567, in simple_launcher raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd) subprocess.CalledProcessError: Command '['D:\code\test\kohya_ss\venv\Scripts\python.exe', 'train_network.py', '--pretrained_model_name_or_path=D:/code/novelai-webui-aki-v3/models/Stable-diffusion/CounterfeitV25_25.safetensors', '--train_data_dir=D:/code/test/train/image', '--resolution=512,512', '--output_dir=D:/code/test/train/model', '--logging_dir=D:/code/test/train/log', '--network_alpha=1', '--save_model_as=safetensors', '--network_module=networks.lora', '--text_encoder_lr=5e-5', '--unet_lr=0.0001', '--network_dim=8', '--output_name=last', '--lr_scheduler_num_cycles=1', '--learning_rate=0.0001', '--lr_scheduler=cosine', '--lr_warmup_steps=230', '--train_batch_size=1', '--max_train_steps=2300', '--save_every_n_epochs=1', '--mixed_precision=fp16', '--save_precision=fp16', '--cache_latents', '--optimizer_type=AdamW8bit', '--max_data_loader_n_workers=0', '--bucket_reso_steps=64', '--xformers', '--bucket_no_upscale']' returned non-zero exit status 1.

bmaltais commented 1 year ago

This error message is a Python traceback, which indicates that an error has occurred while executing a Python program. The error message suggests that there is a problem with loading PyTorch weights from a checkpoint file for the 'openai/clip-vit-large-patch14' model. Specifically, the error message suggests that the PyTorch checkpoint file is not in the expected format or is corrupted, which is preventing the weights from being loaded.

Sansblaze commented 1 year ago

This error message is a Python traceback, which indicates that an error has occurred while executing a Python program. The error message suggests that there is a problem with loading PyTorch weights from a checkpoint file for the 'openai/clip-vit-large-patch14' model. Specifically, the error message suggests that the PyTorch checkpoint file is not in the expected format or is corrupted, which is preventing the weights from being loaded.

I have the exact same error, everything was working fine before i updated.. any fix?

Edit : Deleted .cache\huggingface\hub\models--openai--clip-vit-large-patch14

mlsjla commented 1 year ago

@Sansblaze

Thank u ... I have done..