Linaqruf / kohya-trainer

Adapted from https://note.com/kohya_ss/n/nbf7ce8d80f29 for easier cloning
Apache License 2.0
1.83k stars 300 forks source link

suggest:More reasonable Google folder data copying code #249

Closed wzgrx closed 1 year ago

wzgrx commented 1 year ago

If the folder is on Google Drive, the original copying method is very complex and troublesome You can refer to the following Collab code

https://colab.research.google.com/github/WSH032/lora-scripts/blob/main/Colab_Lora_train.ipynb#scrollTo=j6nkqp9Fb7Dg

@title ####拷贝材料(支持重复训练时选择新的路径)

@markdown 默认给的是教程(三)中的谷歌硬盘目录,你也可以自定义训练集和正则化集路径(只要你知道你在做什么),请选择repeat_concept的父目录

@markdown 为了稳定性和性能,notebook会把材料拷贝到colab的/content/lora-scripts/train/中进行操作

@markdown 是否使用自定义路径,是否拷贝正则化图片

use_data_dir_self = False #@param {type:"boolean"} copy_reg = False #@param {type:"boolean"}

@markdown 自定义训练集路径,正则化集路径(仅在勾选后有效)(不要使用带空格、中文的路径)

train_data_dir_self = "/content/drive/MyDrive/Lora/input/" #@param {type:'string'} reg_data_dir_self = "/content/drive/MyDrive/Lora/reg/" #@param {type:'string'}

if use_data_dir_self: print(f"你使用的是自定义路径") else: train_data_dir_self = "/content/drive/MyDrive/Lora/input/" reg_data_dir_self = "/content/drive/MyDrive/Lora/reg/" print(f"你使用的是默认路径") print(f"训练集地址为:{train_data_dir_self}")

@markdown 拷贝时间由训练材料大小决定

@markdown 出现这样的输出就是正确放置了文件且完成了拷贝

@markdown copy.png

删除之前的训练材料

!mkdir -p /content/lora-scripts/train/ #防止首次运行报错 !rm -r /content/lora-scripts/train/

从谷歌硬盘中拷贝你之前上传的训练材料

print("拷贝训练集中") !mkdir -p {train_data_dir} !cp -r {train_data_dir_self}/* {train_data_dir} !echo "copy训练材料完成."

if copy_reg:

拷贝正则化图片

print(f"正则化集地址为:{reg_data_dir_self}") print("拷贝正则化集中") !mkdir -p {reg_data_dir} !cp -r {reg_data_dir_self}/* {reg_data_dir} !echo "copy正则化图片完成." else: print("不拷贝正则化集")

%cd /content/lora-scripts

Linaqruf commented 1 year ago

Why we need to copy the train data dir? I think you can just mount the drive and copy the drive folder path.