[Dataset 0]
loading image sizes.
100% 42/42 [00:00<00:00, 6428.99it/s]
make buckets
number of images (including repeats) / 各bucketの画像枚数（繰り返し回数を含む）
bucket 0: resolution (1024, 1024), count: 84
mean ar error (without repeats): 0.0
Warning: SDXL has been trained with noise_offset=0.0357, but noise_offset is disabled due to multires_noise_iterations / SDXLはnoise_offset=0.0357で学習されていますが、multires_noise_iterationsが有効になっているためnoise_offsetは無効になります
preparing accelerator
loading model for process 0/1
load Diffusers pretrained models: misri/realismEngineSDXL_v30VAE, variant=fp16
The config attributes {'force_upcast': True} were passed to AutoencoderKL, but are not expected and will be ignored. Please verify your config.json configuration file.
The config attributes {'attention_type': 'default', 'dropout': 0.0} were passed to UNet2DConditionModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Traceback (most recent call last):
File "/content/kohya-trainer/train_network_xl_wrapper.py", line 10, in
trainer.train(args)
File "/content/kohya-trainer/train_network.py", line 213, in train
model_version, text_encoder, vae, unet = self.load_target_model(args, weight_dtype, accelerator)
File "/content/kohya-trainer/sdxl_train_network.py", line 34, in load_target_model
) = sdxl_train_util.load_target_model(args, accelerator, sdxl_model_util.MODEL_VERSION_SDXL_BASE_V0_9, weight_dtype)
File "/content/kohya-trainer/library/sdxl_train_util.py", line 33, in load_target_model
) = _load_target_model(
File "/content/kohya-trainer/library/sdxl_train_util.py", line 79, in _load_target_model
pipe = StableDiffusionXLPipeline.from_pretrained(name_or_path, torch_dtype=weight_dtype, variant=variant, tokenizer=None)
File "/usr/local/lib/python3.10/dist-packages/diffusers/pipelines/pipeline_utils.py", line 1037, in from_pretrained
loaded_sub_model = load_sub_model(
File "/usr/local/lib/python3.10/dist-packages/diffusers/pipelines/pipeline_utils.py", line 450, in load_sub_model
loaded_sub_model = load_method(os.path.join(cached_folder, name), **loading_kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py", line 1825, in from_pretrained
return cls._from_pretrained(
File "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py", line 2044, in _from_pretrained
raise ValueError(
ValueError: Non-consecutive added token '!' found. Should have index 49408 but has index 0 in saved vocabulary.
Please let me know how to process this problem.