bmaltais / kohya_ss

Apache License 2.0
9.54k stars 1.23k forks source link

No subfolders found #547

Closed highjohnconquer closed 8 months ago

highjohnconquer commented 1 year ago

kohya isn't able to find my files for some reason. Also a few more errors:

Validating that requirements are satisfied. All requirements satisfied. Load CSS... Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch(). Save... E:/AI/ToniRizzo.json Save... E:/AI/ToniRizzo.json Save... Removing existing directory E:/Documents/AI/stable-diffusion-webui/models/Lora\img/58_ToniRizzo woman... Copy C:\Users\raziq\OneDrive\Pictures\Toni Driver\Toni_Training\processed to E:/Documents/AI/stable-diffusion-webui/models/Lora\img/58_ToniRizzo woman... Regularization images directory is missing... not copying regularisation images... Done creating kohya_ss training folder structure at E:/Documents/AI/stable-diffusion-webui/models/Lora... Traceback (most recent call last): File "E:\Documents\AI\kohya_ss\venv\lib\site-packages\gradio\routes.py", line 384, in run_predict output = await app.get_blocks().process_api( File "E:\Documents\AI\kohya_ss\venv\lib\site-packages\gradio\blocks.py", line 1024, in process_api result = await self.call_function( File "E:\Documents\AI\kohya_ss\venv\lib\site-packages\gradio\blocks.py", line 836, in call_function prediction = await anyio.to_thread.run_sync( File "E:\Documents\AI\kohya_ss\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "E:\Documents\AI\kohya_ss\venv\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "E:\Documents\AI\kohya_ss\venv\lib\site-packages\anyio_backends_asyncio.py", line 867, in run result = context.run(func, *args) File "E:\Documents\AI\kohya_ss\library\tensorboard_gui.py", line 14, in start_tensorboard if not os.listdir(logging_dir): FileNotFoundError: [WinError 3] The system cannot find the path specified: '' max_train_steps = 0 stop_text_encoder_training = 0 lr_warmup_steps = 0 accelerate launch --num_cpu_threads_per_process=12 "train_network.py" --enable_bucket --pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5" --train_data_dir="C:\Users\raziq\OneDrive\Pictures\Toni Driver\Toni_Training\processed" --resolution=1024,256 --output_dir="E:/Documents/AI/stable-diffusion-webui/models/Lora" --logging_dir="" --network_alpha="128" --save_model_as=safetensors --network_module=networks.lora --text_encoder_lr=5e-5 --unet_lr=0.0001 --network_dim=128 --output_name="ToniRizzo" --lr_scheduler_num_cycles="1" --learning_rate="0.0001" --lr_scheduler="cosine_with_restarts" --train_batch_size="1" --save_every_n_epochs="1" --mixed_precision="fp16" --save_precision="fp16" --cache_latents --optimizer_type="AdamW8bit" --max_data_loader_n_workers="0" --bucket_reso_steps=64 --xformers --bucket_no_upscale Could not find module 'E:\Documents\AI\kohya_ss\venv\Lib\site-packages\xformers_C.pyd' (or one of its dependencies). Try using the full path with constructor syntax. WARNING:root:WARNING: Could not find module 'E:\Documents\AI\kohya_ss\venv\Lib\site-packages\xformers_C.pyd' (or one of its dependencies). Try using the full path with constructor syntax. Need to compile C++ extensions to get sparse attention suport. Please run python setup.py build develop prepare tokenizer Use DreamBooth method. prepare images. 0 train images with repeating. 0 reg images. no regularization images / 正則化画像が見つかりませんでした [Dataset 0] batch_size: 1 resolution: (1024, 256) enable_bucket: True min_bucket_reso: 256 max_bucket_reso: 1024 bucket_reso_steps: 64 bucket_no_upscale: True

[Dataset 0] loading image sizes. 0it [00:00, ?it/s] make buckets min_bucket_reso and max_bucket_reso are ignored if bucket_no_upscale is set, because bucket reso is defined by image size automatically / bucket_no_upscaleが指定された場合は、bucketの解像度は画 像サイズから自動計算されるため、min_bucket_resoとmax_bucket_resoは無視されます number of images (including repeats) / 各bucketの画像枚数(繰り返し回数を含む) E:\Documents\AI\kohya_ss\venv\lib\site-packages\numpy\core\fromnumeric.py:3464: RuntimeWarning: Mean of empty slice. return _methods._mean(a, axis=axis, dtype=dtype, E:\Documents\AI\kohya_ss\venv\lib\site-packages\numpy\core_methods.py:192: RuntimeWarning: invalid value encountered in scalar divide ret = ret.dtype.type(ret / rcount) mean ar error (without repeats): nan No data found. Please verify arguments (train_data_dir must be the parent of folders with images) / 画像がありません。引数指定を確認してください(train_data_dirには画像があるフォルダではなく、画像があるフォルダの親フォルダを指定する必要があります) No subfolders were found in C:\Users\raziq\OneDrive\Pictures\Toni Driver\Toni_Training\processed can't train...033[0m No subfolders were found in C:\Users\raziq\OneDrive\Pictures\Toni Driver\Toni_Training\processed can't train...033[0m No subfolders were found in C:\Users\raziq\OneDrive\Pictures\Toni Driver\Toni_Training\processed can't train...033[0m No subfolders were found in E:\Pictures\ToniRizzo Training can't train...033[0m No subfolders were found in E:\Pictures\ToniRizzo Training can't train...033[0m No subfolders were found in E:\Pictures\ToniRizzoTraining can't train...033[0m Removing existing directory E:/Documents/AI/stable-diffusion-webui/models/Lora\img/58_ToniRizzo woman... Copy E:\Pictures\ToniRizzoTraining to E:/Documents/AI/stable-diffusion-webui/models/Lora\img/58_ToniRizzo woman... Regularization images directory is missing... not copying regularisation images... Done creating kohya_ss training folder structure at E:/Documents/AI/stable-diffusion-webui/models/Lora... No subfolders were found in E:\Pictures\ToniRizzoTraining can't train...033[0m

bmaltais commented 1 year ago

Are images in the folder matching one of these file types: '.jpg', '.jpeg', '.png', '.gif', '.webp'

highjohnconquer commented 1 year ago

they are jpg and png,

i moved the files around and now i get this error:

←[33mNo subfolders were found in E:\Documents\AI\stable-diffusion-webui\models\Lora\img\58_ToniRizzo woman can't train...033[0m ←[33mSubfolder img does not have a proper repeat value, please correct the name or remove it... can't train...←[0m ←[33mSubfolder log does not have a proper repeat value, please correct the name or remove it... can't train...←[0m ←[33mSubfolder model does not have a proper repeat value, please correct the name or remove it... can't train...←[0m ←[33mSubfolder reg does not have a proper repeat value, please correct the name or remove it... can't train...←[0m ←[33mSubfolder sample does not have a proper repeat value, please correct the name or remove it... can't train...←[0m ←[33mNo images were found in folder E:/Documents/AI/stable-diffusion-webui/models/Lora ... please rectify!←[0m

johndpope commented 1 year ago
# Drambootd, Lora and TI image folder structure

To ensure successful training with Kohya, it is crucial to follow a specific folder structure that provides the necessary image repeats. Please adhere to the following structure precisely:

Folder Structure Example:

```txt
c:
|
├──images
|   |
|   ├── 30_cat
|   |   |
|   |   ├── image1.jpg
|   |   ├── image1.txt
|   |   ├── image2.png
|   |   └── image2.txt
|   |
|   ├── 30_dog
|   |   |
|   |   ├── image1.jpg
|   |   ├── image1.txt
|   |   ├── image2.png
|   |   └── image2.txt
|   |
|   └── 40_black mamba
|       |
|       ├── image1.jpg
|       ├── image1.txt
|       ├── image2.png
|       └── image2.txt
|
├──regularization
|   |
|   ├── 1_cat
|   |   |
|   |   ├── reg1.jpg
|   |   ├── reg2.jpg
|   |
|   ├── 1_dog
|   |   |
|   |   ├── reg1.jpg
|   |   ├── reg2.jpg
|   |
|   └── 1_black mamba
|       |
|       ├── reg1.jpg
|       ├── reg2.jpg

Please note the following important information regarding file extensions and their impact on concept names during model training:

If a file with a .txt or .caption extension and the same name as an image is present in the image subfolder, it will take precedence over the concept name during the model training process. For example, if there is an image file named image1.jpg in the 30_cat subfolder, and there is a corresponding text file named image1.txt or image1.caption in the same subfolder, the concept name used during training will be determined by the content of that text file rather than the subfolder name.

Ensure that the content of such text files accurately reflects the desired concept name or any relevant caption information associated with the corresponding image.

By considering this information and maintaining the proper folder structure, including any necessary text or caption files, you can ensure a smooth and effective training process with Kohya.


for me - my folder structure looks like this.
```shell
/rover
.
└── 01_marsrover
    ├── 10.png
    ├── 10.txt
    ├── 11.png
    ├── 11.txt
    ├── 12
    ├── 12.txt
    ├── 1.png
    ├── 1.txt
    ├── 2.png
    ├── 2.txt
    ├── 3.jpeg
    ├── 3.txt
    ├── 4.jpg
    ├── 4.txt
    ├── 5.png
    ├── 5.txt
    ├── 6.png
    ├── 6.txt
    ├── 7.png
    ├── 7.txt
    ├── 8.png
    ├── 8.txt
    ├── 9.png
    └── 9.txt

1 directory, 24 files
(torch2) ➜  rover 
Linjianfengccc commented 11 months ago

this rule is so weird, took me so much time..

johndpope commented 11 months ago

did you get it working? my rover example fails Screenshot from 2023-11-13 11-29-01

Linjianfengccc commented 11 months ago

@johndpope you should select rover folder as the source folder, but not 01marsrover. The number in format of \<number>\<name> stands for train steps, in your case, the folder name can be 10_marsrover, 30_marsrover, or something else like this, up to your need