Anothergazz
commented
1 month ago I would like to add that I have now done a new instal, twice, and I believe I have the exact same issue as pianogospel. Here is what the command window shows
(venv) C:\AI\ai-toolkit>python run.py config/JL_lora_flux_24gb.yaml
Running 1 job
C:\AI\ai-toolkit\venv\lib\site-packages\albumentations__init__.py:13: UserWarning: A new version of Albumentations is available: 1.4.18 (you have 1.4.15). Upgrade using: pip install -U albumentations. To disable automatic update checks, set the environment variable NO_ALBUMENTATIONS_UPDATE to 1.
check_for_updates()
C:\AI\ai-toolkit\venv\lib\site-packages\controlnet_aux\mediapipe_face\mediapipe_face_common.py:7: UserWarning: The module 'mediapipe' is not installed. The package will have limited functionality. Please install it using the command: pip install 'mediapipe'
warnings.warn(
C:\AI\ai-toolkit\venv\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_5m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_5m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\AI\ai-toolkit\venv\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_11m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_11m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\AI\ai-toolkit\venv\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\AI\ai-toolkit\venv\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_384 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_384. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\AI\ai-toolkit\venv\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_512 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_512. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
{
"type": "sd_trainer",
"training_folder": "output",
"device": "cuda:0",
"network": {
"type": "lora",
"linear": 16,
"linear_alpha": 16
},
"save": {
"dtype": "float16",
"save_every": 250,
"max_step_saves_to_keep": 4,
"push_to_hub": false
},
"datasets": [
{
"folder_path": "C:\AI\Lora\jennytestFlux",
"caption_ext": "txt",
"caption_dropout_rate": 0.05,
"shuffle_tokens": false,
"cache_latents_to_disk": true,
"resolution": [
512,
768,
1024
]
}
],
"train": {
"batch_size": 1,
"steps": 2000,
"gradient_accumulation_steps": 1,
"train_unet": true,
"train_text_encoder": false,
"gradient_checkpointing": true,
"noise_scheduler": "flowmatch",
"optimizer": "adamw8bit",
"lr": 0.0001,
"ema_config": {
"use_ema": true,
"ema_decay": 0.99
},
"dtype": "bf16"
},
"model": {
"name_or_path": "black-forest-labs/FLUX.1-dev",
"is_flux": true,
"quantize": true,
"low_vram": true
},
"sample": {
"sampler": "flowmatch",
"sample_every": 250,
"width": 1024,
"height": 1024,
"prompts": [
"JL, a woman holding a coffee cup, in a beanie, sitting at a cafe",
"a woman holding a coffee cup, in a beanie, sitting at a cafe"
],
"neg": "",
"seed": 42,
"walk_seed": true,
"guidance_scale": 4,
"sample_steps": 20
}
}
Using EMA
C:\AI\ai-toolkit\extensions_built_in\sd_trainer\SDTrainer.py:61: FutureWarning: torch.cuda.amp.GradScaler(args...) is deprecated. Please use torch.amp.GradScaler('cuda', args...) instead.
self.scaler = torch.cuda.amp.GradScaler()
#############################################
Running job: JL_flux_lora_v1
#############################################
Running 1 process Loading Flux model Loading transformer Quantizing transformer C:\AI\ai-toolkit\venv\lib\site-packages\torch\utils\cpp_extension.py:380: UserWarning: Error checking compiler version for cl: [WinError 2] The system cannot find the file specified warnings.warn(f'Error checking compiler version for {compiler}: {error}') C:\AI\ai-toolkit\venv\lib\site-packages\torch\utils\cpp_extension.py:1965: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST']. warnings.warn( INFO: Could not find files for the given pattern(s). Error running job: Command '['where', 'cl']' returned non-zero exit status 1.
======================================== Result:
- 0 completed jobs
-
1 failure
Traceback (most recent call last): File "C:\AI\ai-toolkit\run.py", line 90, in
main() File "C:\AI\ai-toolkit\run.py", line 86, in main raise e File "C:\AI\ai-toolkit\run.py", line 78, in main job.run() File "C:\AI\ai-toolkit\jobs\ExtensionJob.py", line 22, in run process.run() File "C:\AI\ai-toolkit\jobs\process\BaseSDTrainProcess.py", line 1241, in run self.sd.load_model() File "C:\AI\ai-toolkit\toolkit\stable_diffusion_model.py", line 613, in load_model transformer.to(self.device_torch) File "C:\AI\ai-toolkit\venv\lib\site-packages\torch\nn\modules\module.py", line 1174, in to return self._apply(convert) File "C:\AI\ai-toolkit\venv\lib\site-packages\torch\nn\modules\module.py", line 780, in _apply module._apply(fn) File "C:\AI\ai-toolkit\venv\lib\site-packages\torch\nn\modules\module.py", line 780, in _apply module._apply(fn) File "C:\AI\ai-toolkit\venv\lib\site-packages\torch\nn\modules\module.py", line 780, in _apply module._apply(fn) File "C:\AI\ai-toolkit\venv\lib\site-packages\torch\nn\modules\module.py", line 805, in _apply param_applied = fn(param) File "C:\AI\ai-toolkit\venv\lib\site-packages\torch\nn\modules\module.py", line 1160, in convert return t.to( File "C:\AI\ai-toolkit\venv\lib\site-packages\optimum\quanto\tensor\weights\qbytes.py", line 273, in torch_function return func(*args, **kwargs) File "C:\AI\ai-toolkit\venv\lib\site-packages\optimum\quanto\tensor\weights\qbytes.py", line 299, in torch_dispatch return WeightQBytesTensor.create( File "C:\AI\ai-toolkit\venv\lib\site-packages\optimum\quanto\tensor\weights\qbytes.py", line 140, in create return MarlinF8QBytesTensor(qtype, axis, size, stride, data, scale, requires_grad) File "C:\AI\ai-toolkit\venv\lib\site-packages\optimum\quanto\tensor\weights\marlin\fp8\qbits.py", line 80, in init data_packed = MarlinF8PackedTensor.pack(data) # pack fp8 data to in32, and apply marlier re-ordering. File "C:\AI\ai-toolkit\venv\lib\site-packages\optimum\quanto\tensor\weights\marlin\fp8\packed.py", line 183, in pack data_int32 = torch.ops.quanto.pack_fp8marlin( File "C:\AI\ai-toolkit\venv\lib\site-packages\torch_ops.py", line 1061, in call return self._op(*args, *(kwargs or {})) File "C:\AI\ai-toolkit\venv\lib\site-packages\optimum\quanto\library\extensions\cuda__init__.py", line 164, in gptq_marlin_repack return ext.lib.gptq_marlin_repack(b_q_weight, perm, size_k, size_n, num_bits) File "C:\AI\ai-toolkit\venv\lib\site-packages\optimum\quanto\library\extensions\extension.py", line 42, in lib self._lib = load( File "C:\AI\ai-toolkit\venv\lib\site-packages\torch\utils\cpp_extension.py", line 1312, in load return _jit_compile( File "C:\AI\ai-toolkit\venv\lib\site-packages\torch\utils\cpp_extension.py", line 1722, in _jit_compile _write_ninja_file_and_build_library( File "C:\AI\ai-toolkit\venv\lib\site-packages\torch\utils\cpp_extension.py", line 1821, in _write_ninja_file_and_build_library _write_ninja_file_to_build_library( File "C:\AI\ai-toolkit\venv\lib\site-packages\torch\utils\cpp_extension.py", line 2246, in _write_ninja_file_to_build_library _write_ninja_file( File "C:\AI\ai-toolkit\venv\lib\site-packages\torch\utils\cpp_extension.py", line 2382, in _write_ninja_file cl_paths = subprocess.check_output(['where', File "C:\Users\MrUSB\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 421, in check_output return run(popenargs, stdout=PIPE, timeout=timeout, check=True, File "C:\Users\MrUSB\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['where', 'cl']' returned non-zero exit status 1.
spezialspezial
pianogospel
Astilen
nebooz
sanctimon
inflamously
ironico
dene-
hben35096
I need help, I tried to get help in DISCORD but nobody answered, I already use ai toolkit since august, it works flawlesly. I tried to install the newer version in windows (not an update, a newer installation from the beggining) and I followed all the steps but it simply doesn't work and shows multiple error messages. Can anyone tell me what is the pre requisites for the latest version of ai toolkit? I have Python 3.10.11, nvidia toolkit cuda_11.8.r11.8 , visual studio 2022, cl.exe in environment variable, but it doesn't work. Thanks for any help.