LarryJane491 / Lora-Training-in-Comfy

This custom node lets you train LoRA directly in ComfyUI!
364 stars 50 forks source link

Failed building wheel for xformers, tokenizers #38

Open dhrider001 opened 5 months ago

dhrider001 commented 5 months ago

Hi Larry, thank you for sharing this great node with us! Unfortunately I'm having a bit of trouble getting the requirements to run. No matter what I do, it always fails building wheel for xformers and tokenizers which are required to install pyproject.toml-based projects. I have really tried a lot, including the suggested installation of rust compiler, but unfortunately without success. As I would really like to try out your node, I would really appreciate your help. Many thanks in advance!

My console log/error (tried to cut it as short as possible):

...

Using cached zipp-3.18.1-py3-none-any.whl (8.2 kB) Using cached smmap-5.0.1-py3-none-any.whl (24 kB) Building wheels for collected packages: xformers, tokenizers Building wheel for xformers (setup.py) ... error error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [249 lines of output] fatal: not a git repository (or any of the parent directories): .git running bdist_wheel running build running build_py creating build creating build\lib.win-amd64-cpython-312 creating build\lib.win-amd64-cpython-312\xformers copying xformers\attn_bias_utils.py -> build\lib.win-amd64-cpython-312\xformers copying xformers\checkpoint.py -> build\lib.win-amd64-cpython-312\xformers copying xformers\info.py -> build\lib.win-amd64-cpython-312\xformers copying xformers\test.py -> build\lib.win-amd64-cpython-312\xformers copying xformers\utils.py -> build\lib.win-amd64-cpython-312\xformers copying xformers_cpp_lib.py -> build\lib.win-amd64-cpython-312\xformers copying xformers_deprecation_warning.py -> build\lib.win-amd64-cpython-312\xformers copying xformers__init__.py -> build\lib.win-amd64-cpython-312\xformers creating build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_attn_decoding.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_blocksparse_transformers.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_causal_blocksparse.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_core.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_indexing.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_mem_eff_attention.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_mem_eff_attention_mqa.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_mem_eff_attn_decoder.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_mlp.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_multi_head_dispatch.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_nystrom_utils.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_revnet.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_sddmm.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_sequence_parallel_fused.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_sp24.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_swiglu.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_tiled_matmul.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_transformer.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_triton_blocksparse.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_triton_dropout.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_triton_fused_linear.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_triton_layernorm.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\benchmark_triton_softmax.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks\utils.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks copying xformers\benchmarks__init__.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks creating build\lib.win-amd64-cpython-312\xformers\components copying xformers\components\activations.py -> build\lib.win-amd64-cpython-312\xformers\components copying xformers\components\input_projection.py -> build\lib.win-amd64-cpython-312\xformers\components copying xformers\components\multi_head_dispatch.py -> build\lib.win-amd64-cpython-312\xformers\components copying xformers\components\patch_embedding.py -> build\lib.win-amd64-cpython-312\xformers\components copying xformers\components\residual.py -> build\lib.win-amd64-cpython-312\xformers\components copying xformers\components\reversible.py -> build\lib.win-amd64-cpython-312\xformers\components copying xformers\components\simplicial_embedding.py -> build\lib.win-amd64-cpython-312\xformers\components copying xformers\components__init.py -> build\lib.win-amd64-cpython-312\xformers\components creating build\lib.win-amd64-cpython-312\xformers\factory copying xformers\factory\block_configs.py -> build\lib.win-amd64-cpython-312\xformers\factory copying xformers\factory\block_factory.py -> build\lib.win-amd64-cpython-312\xformers\factory copying xformers\factory\hydra_helper.py -> build\lib.win-amd64-cpython-312\xformers\factory copying xformers\factory\model_factory.py -> build\lib.win-amd64-cpython-312\xformers\factory copying xformers\factory\weight_init.py -> build\lib.win-amd64-cpython-312\xformers\factory copying xformers\factory__init.py -> build\lib.win-amd64-cpython-312\xformers\factory creating build\lib.win-amd64-cpython-312\xformers\helpers copying xformers\helpers\hierarchical_configs.py -> build\lib.win-amd64-cpython-312\xformers\helpers copying xformers\helpers\test_utils.py -> build\lib.win-amd64-cpython-312\xformers\helpers copying xformers\helpers\timm_sparse_attention.py -> build\lib.win-amd64-cpython-312\xformers\helpers copying xformers\helpers__init.py -> build\lib.win-amd64-cpython-312\xformers\helpers creating build\lib.win-amd64-cpython-312\xformers\ops copying xformers\ops\common.py -> build\lib.win-amd64-cpython-312\xformers\ops copying xformers\ops\differentiable_collectives.py -> build\lib.win-amd64-cpython-312\xformers\ops copying xformers\ops\indexing.py -> build\lib.win-amd64-cpython-312\xformers\ops copying xformers\ops\ipc.py -> build\lib.win-amd64-cpython-312\xformers\ops copying xformers\ops\modpar_layers.py -> build\lib.win-amd64-cpython-312\xformers\ops copying xformers\ops\rmsnorm.py -> build\lib.win-amd64-cpython-312\xformers\ops copying xformers\ops\rope_padded.py -> build\lib.win-amd64-cpython-312\xformers\ops copying xformers\ops\seqpar.py -> build\lib.win-amd64-cpython-312\xformers\ops copying xformers\ops\sequence_parallel_fused_ops.py -> build\lib.win-amd64-cpython-312\xformers\ops copying xformers\ops\sp24.py -> build\lib.win-amd64-cpython-312\xformers\ops copying xformers\ops\swiglu_op.py -> build\lib.win-amd64-cpython-312\xformers\ops copying xformers\ops\tiled_matmul.py -> build\lib.win-amd64-cpython-312\xformers\ops copying xformers\ops\unbind.py -> build\lib.win-amd64-cpython-312\xformers\ops copying xformers\ops__init.py -> build\lib.win-amd64-cpython-312\xformers\ops creating build\lib.win-amd64-cpython-312\xformers\profiler copying xformers\profiler\api.py -> build\lib.win-amd64-cpython-312\xformers\profiler copying xformers\profiler\device_limits.py -> build\lib.win-amd64-cpython-312\xformers\profiler copying xformers\profiler\profiler.py -> build\lib.win-amd64-cpython-312\xformers\profiler copying xformers\profiler\profiler_dcgm.py -> build\lib.win-amd64-cpython-312\xformers\profiler copying xformers\profiler\profiler_dcgm_impl.py -> build\lib.win-amd64-cpython-312\xformers\profiler copying xformers\profiler\slow_ops_profiler.py -> build\lib.win-amd64-cpython-312\xformers\profiler copying xformers\profiler__init.py -> build\lib.win-amd64-cpython-312\xformers\profiler creating build\lib.win-amd64-cpython-312\xformers\sparse copying xformers\sparse\blocksparse_tensor.py -> build\lib.win-amd64-cpython-312\xformers\sparse copying xformers\sparse\csr_tensor.py -> build\lib.win-amd64-cpython-312\xformers\sparse copying xformers\sparse\utils.py -> build\lib.win-amd64-cpython-312\xformers\sparse copying xformers\sparse_csr_ops.py -> build\lib.win-amd64-cpython-312\xformers\sparse copying xformers\sparse__init.py -> build\lib.win-amd64-cpython-312\xformers\sparse creating build\lib.win-amd64-cpython-312\xformers\triton copying xformers\triton\dropout.py -> build\lib.win-amd64-cpython-312\xformers\triton copying xformers\triton\fused_linear_layer.py -> build\lib.win-amd64-cpython-312\xformers\triton copying xformers\triton\k_activations.py -> build\lib.win-amd64-cpython-312\xformers\triton copying xformers\triton\k_dropout.py -> build\lib.win-amd64-cpython-312\xformers\triton copying xformers\triton\k_fused_matmul_bw.py -> build\lib.win-amd64-cpython-312\xformers\triton copying xformers\triton\k_fused_matmul_fw.py -> build\lib.win-amd64-cpython-312\xformers\triton copying xformers\triton\k_layer_norm.py -> build\lib.win-amd64-cpython-312\xformers\triton copying xformers\triton\k_softmax.py -> build\lib.win-amd64-cpython-312\xformers\triton copying xformers\triton\layer_norm.py -> build\lib.win-amd64-cpython-312\xformers\triton copying xformers\triton\softmax.py -> build\lib.win-amd64-cpython-312\xformers\triton copying xformers\triton\vararg_kernel.py -> build\lib.win-amd64-cpython-312\xformers\triton copying xformers\triton__init.py -> build\lib.win-amd64-cpython-312\xformers\triton creating build\lib.win-amd64-cpython-312\xformers_flash_attn copying xformers_flash_attn\bert_padding.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn copying xformers_flash_attn\flash_attn_interface.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn copying xformers_flash_attn\flash_attn_triton.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn copying xformers_flash_attn\flash_attn_triton_og.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn copying xformers_flash_attn\flash_blocksparse_attention.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn copying xformers_flash_attn\flash_blocksparse_attn_interface.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn copying xformers_flash_attn\fused_softmax.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn copying xformers_flash_attn__init.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn creating build\lib.win-amd64-cpython-312\xformers\benchmarks\LRA copying xformers\benchmarks\LRA\batch_fetch_results.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks\LRA copying xformers\benchmarks\LRA\batch_submit.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks\LRA copying xformers\benchmarks\LRA\run_grid_search.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks\LRA copying xformers\benchmarks\LRA\run_tasks.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks\LRA copying xformers\benchmarks\LRA\run_with_submitit.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks\LRA copying xformers\benchmarks\LRA__init__.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks\LRA creating build\lib.win-amd64-cpython-312\xformers\benchmarks\LRA\code copying xformers\benchmarks\LRA\code\dataset.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks\LRA\code copying xformers\benchmarks\LRA\code\model_wrapper.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks\LRA\code copying xformers\benchmarks\LRA\code\init.py -> build\lib.win-amd64-cpython-312\xformers\benchmarks\LRA\code creating build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\attention_mask.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\attention_patterns.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\base.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\blocksparse.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\compositional.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\core.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\favor.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\fourier_mix.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\global_tokens.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\lambda_layer.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\linformer.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\local.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\nystrom.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\ortho.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\pooling.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\random.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\scaled_dot_product.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\sparsity_config.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\utils.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\visual.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention_sputnik_sparse.py -> build\lib.win-amd64-cpython-312\xformers\components\attention copying xformers\components\attention\init.py -> build\lib.win-amd64-cpython-312\xformers\components\attention creating build\lib.win-amd64-cpython-312\xformers\components\feedforward copying xformers\components\feedforward\base.py -> build\lib.win-amd64-cpython-312\xformers\components\feedforward copying xformers\components\feedforward\conv_mlp.py -> build\lib.win-amd64-cpython-312\xformers\components\feedforward copying xformers\components\feedforward\fused_mlp.py -> build\lib.win-amd64-cpython-312\xformers\components\feedforward copying xformers\components\feedforward\mixture_of_experts.py -> build\lib.win-amd64-cpython-312\xformers\components\feedforward copying xformers\components\feedforward\mlp.py -> build\lib.win-amd64-cpython-312\xformers\components\feedforward copying xformers\components\feedforward\init.py -> build\lib.win-amd64-cpython-312\xformers\components\feedforward creating build\lib.win-amd64-cpython-312\xformers\components\positional_embedding copying xformers\components\positional_embedding\base.py -> build\lib.win-amd64-cpython-312\xformers\components\positional_embedding copying xformers\components\positional_embedding\param.py -> build\lib.win-amd64-cpython-312\xformers\components\positional_embedding copying xformers\components\positional_embedding\rotary.py -> build\lib.win-amd64-cpython-312\xformers\components\positional_embedding copying xformers\components\positional_embedding\sine.py -> build\lib.win-amd64-cpython-312\xformers\components\positional_embedding copying xformers\components\positional_embedding\vocab.py -> build\lib.win-amd64-cpython-312\xformers\components\positional_embedding copying xformers\components\positional_embedding\init__.py -> build\lib.win-amd64-cpython-312\xformers\components\positional_embedding creating build\lib.win-amd64-cpython-312\xformers\components\attention\feature_maps copying xformers\components\attention\feature_maps\base.py -> build\lib.win-amd64-cpython-312\xformers\components\attention\feature_maps copying xformers\components\attention\feature_maps\softmax.py -> build\lib.win-amd64-cpython-312\xformers\components\attention\feature_maps copying xformers\components\attention\feature_maps\init.py -> build\lib.win-amd64-cpython-312\xformers\components\attention\feature_maps creating build\lib.win-amd64-cpython-312\xformers\ops\fmha copying xformers\ops\fmha\attn_bias.py -> build\lib.win-amd64-cpython-312\xformers\ops\fmha copying xformers\ops\fmha\ck.py -> build\lib.win-amd64-cpython-312\xformers\ops\fmha copying xformers\ops\fmha\ck_decoder.py -> build\lib.win-amd64-cpython-312\xformers\ops\fmha copying xformers\ops\fmha\ck_splitk.py -> build\lib.win-amd64-cpython-312\xformers\ops\fmha copying xformers\ops\fmha\common.py -> build\lib.win-amd64-cpython-312\xformers\ops\fmha copying xformers\ops\fmha\cutlass.py -> build\lib.win-amd64-cpython-312\xformers\ops\fmha copying xformers\ops\fmha\decoder.py -> build\lib.win-amd64-cpython-312\xformers\ops\fmha copying xformers\ops\fmha\dispatch.py -> build\lib.win-amd64-cpython-312\xformers\ops\fmha copying xformers\ops\fmha\flash.py -> build\lib.win-amd64-cpython-312\xformers\ops\fmha copying xformers\ops\fmha\small_k.py -> build\lib.win-amd64-cpython-312\xformers\ops\fmha copying xformers\ops\fmha\triton_splitk.py -> build\lib.win-amd64-cpython-312\xformers\ops\fmha copying xformers\ops\fmha\init__.py -> build\lib.win-amd64-cpython-312\xformers\ops\fmha creating build\lib.win-amd64-cpython-312\xformers\ops_triton copying xformers\ops_triton\k_index_select_cat.py -> build\lib.win-amd64-cpython-312\xformers\ops_triton copying xformers\ops_triton\k_scaled_index_add.py -> build\lib.win-amd64-cpython-312\xformers\ops_triton copying xformers\ops_triton\rmsnorm_kernels.py -> build\lib.win-amd64-cpython-312\xformers\ops_triton copying xformers\ops_triton\rope_padded_kernels.py -> build\lib.win-amd64-cpython-312\xformers\ops_triton copying xformers\ops_triton\sequence_parallel_fused_kernels.py -> build\lib.win-amd64-cpython-312\xformers\ops_triton copying xformers\ops_triton\tiled_matmul_kernels.py -> build\lib.win-amd64-cpython-312\xformers\ops_triton copying xformers\ops_triton\init__.py -> build\lib.win-amd64-cpython-312\xformers\ops_triton creating build\lib.win-amd64-cpython-312\xformers_flash_attn\layers copying xformers_flash_attn\layers\patch_embed.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\layers copying xformers_flash_attn\layers\rotary.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\layers copying xformers_flash_attn\layers\init__.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\layers creating build\lib.win-amd64-cpython-312\xformers_flash_attn\losses copying xformers_flash_attn\losses\cross_entropy.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\losses copying xformers_flash_attn\losses\init__.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\losses creating build\lib.win-amd64-cpython-312\xformers_flash_attn\models copying xformers_flash_attn\models\baichuan.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\models copying xformers_flash_attn\models\bert.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\models copying xformers_flash_attn\models\bigcode.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\models copying xformers_flash_attn\models\btlm.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\models copying xformers_flash_attn\models\falcon.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\models copying xformers_flash_attn\models\gpt.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\models copying xformers_flash_attn\models\gptj.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\models copying xformers_flash_attn\models\gpt_neox.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\models copying xformers_flash_attn\models\llama.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\models copying xformers_flash_attn\models\opt.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\models copying xformers_flash_attn\models\vit.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\models copying xformers_flash_attn\models\init.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\models creating build\lib.win-amd64-cpython-312\xformers_flash_attn\modules copying xformers_flash_attn\modules\block.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\modules copying xformers_flash_attn\modules\embedding.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\modules copying xformers_flash_attn\modules\mha.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\modules copying xformers_flash_attn\modules\mlp.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\modules copying xformers_flash_attn\modules\init__.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\modules creating build\lib.win-amd64-cpython-312\xformers_flash_attn\ops copying xformers_flash_attn\ops\activations.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\ops copying xformers_flash_attn\ops\fused_dense.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\ops copying xformers_flash_attn\ops\layer_norm.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\ops copying xformers_flash_attn\ops\rms_norm.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\ops copying xformers_flash_attn\ops\init.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\ops creating build\lib.win-amd64-cpython-312\xformers_flash_attn\utils copying xformers_flash_attn\utils\benchmark.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\utils copying xformers_flash_attn\utils\distributed.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\utils copying xformers_flash_attn\utils\generation.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\utils copying xformers_flash_attn\utils\pretrained.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\utils copying xformers_flash_attn\utils\init__.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\utils creating build\lib.win-amd64-cpython-312\xformers_flash_attn\ops\triton copying xformers_flash_attn\ops\triton\cross_entropy.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\ops\triton copying xformers_flash_attn\ops\triton\k_activations.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\ops\triton copying xformers_flash_attn\ops\triton\layer_norm.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\ops\triton copying xformers_flash_attn\ops\triton\linear.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\ops\triton copying xformers_flash_attn\ops\triton\mlp.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\ops\triton copying xformers_flash_attn\ops\triton\rotary.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\ops\triton copying xformers_flash_attn\ops\triton\init__.py -> build\lib.win-amd64-cpython-312\xformers_flash_attn\ops\triton running build_ext C:\Users\Daily Use\AppData\Local\Programs\Python\Python312\Lib\site-packages\torch\utils\cpp_extension.py:381: UserWarning: Error checking compiler version for cl: [WinError 2] Das System kann die angegebene Datei nicht finden warnings.warn(f'Error checking compiler version for {compiler}: {error}') C:\Users\Daily Use\AppData\Local\Programs\Python\Python312\Lib\site-packages\torch\utils\cpp_extension.py:415: UserWarning: The detected CUDA version (12.4) has a minor version mismatch with the version that was used to compile PyTorch (12.1). Most likely this shouldn't be a problem. warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda)) building 'xformers._C_flashattention' extension creating C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\xformers_af86847e30fe4a0c83ceb26d8a08709c\build\temp.win-amd64-cpython-312 creating C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\xformers_af86847e30fe4a0c83ceb26d8a08709c\build\temp.win-amd64-cpython-312\Release creating C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\xformers_af86847e30fe4a0c83ceb26d8a08709c\build\temp.win-amd64-cpython-312\Release\Users creating C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\xformers_af86847e30fe4a0c83ceb26d8a08709c\build\temp.win-amd64-cpython-312\Release\Users\Daily Use creating C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\xformers_af86847e30fe4a0c83ceb26d8a08709c\build\temp.win-amd64-cpython-312\Release\Users\Daily Use\AppData creating C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\xformers_af86847e30fe4a0c83ceb26d8a08709c\build\temp.win-amd64-cpython-312\Release\Users\Daily Use\AppData\Local creating C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\xformers_af86847e30fe4a0c83ceb26d8a08709c\build\temp.win-amd64-cpython-312\Release\Users\Daily Use\AppData\Local\Temp creating C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\xformers_af86847e30fe4a0c83ceb26d8a08709c\build\temp.win-amd64-cpython-312\Release\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l creating C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\xformers_af86847e30fe4a0c83ceb26d8a08709c\build\temp.win-amd64-cpython-312\Release\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\xformers_af86847e30fe4a0c83ceb26d8a08709c creating C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\xformers_af86847e30fe4a0c83ceb26d8a08709c\build\temp.win-amd64-cpython-312\Release\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\xformers_af86847e30fe4a0c83ceb26d8a08709c\third_party error: could not create 'C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\xformers_af86847e30fe4a0c83ceb26d8a08709c\build\temp.win-amd64-cpython-312\Release\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\xformers_af86847e30fe4a0c83ceb26d8a08709c\third_party': Der Dateiname oder die Erweiterung ist zu lang [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for xformers Running setup.py clean for xformers Building wheel for tokenizers (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for tokenizers (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [593 lines of output] running bdist_wheel running build running build_py creating build creating build\lib.win-amd64-cpython-312 creating build\lib.win-amd64-cpython-312\tokenizers copying py_src\tokenizers__init.py -> build\lib.win-amd64-cpython-312\tokenizers creating build\lib.win-amd64-cpython-312\tokenizers\models copying py_src\tokenizers\models__init.py -> build\lib.win-amd64-cpython-312\tokenizers\models creating build\lib.win-amd64-cpython-312\tokenizers\decoders copying py_src\tokenizers\decoders__init.py -> build\lib.win-amd64-cpython-312\tokenizers\decoders creating build\lib.win-amd64-cpython-312\tokenizers\normalizers copying py_src\tokenizers\normalizers__init.py -> build\lib.win-amd64-cpython-312\tokenizers\normalizers creating build\lib.win-amd64-cpython-312\tokenizers\pre_tokenizers copying py_src\tokenizers\pre_tokenizers__init.py -> build\lib.win-amd64-cpython-312\tokenizers\pre_tokenizers creating build\lib.win-amd64-cpython-312\tokenizers\processors copying py_src\tokenizers\processors__init.py -> build\lib.win-amd64-cpython-312\tokenizers\processors creating build\lib.win-amd64-cpython-312\tokenizers\trainers copying py_src\tokenizers\trainers__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\trainers creating build\lib.win-amd64-cpython-312\tokenizers\implementations copying py_src\tokenizers\implementations\base_tokenizer.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations copying py_src\tokenizers\implementations\bert_wordpiece.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations copying py_src\tokenizers\implementations\byte_level_bpe.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations copying py_src\tokenizers\implementations\char_level_bpe.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations copying py_src\tokenizers\implementations\sentencepiece_bpe.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations copying py_src\tokenizers\implementations\sentencepiece_unigram.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations copying py_src\tokenizers\implementations\init.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations creating build\lib.win-amd64-cpython-312\tokenizers\tools copying py_src\tokenizers\tools\visualizer.py -> build\lib.win-amd64-cpython-312\tokenizers\tools copying py_src\tokenizers\tools\init__.py -> build\lib.win-amd64-cpython-312\tokenizers\tools copying py_src\tokenizers\init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers copying py_src\tokenizers\models\init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\models copying py_src\tokenizers\decoders\init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\decoders copying py_src\tokenizers\normalizers\init.pyi -> build\lib.win-amd64-cpython-312\tokenizers\normalizers copying py_src\tokenizers\pre_tokenizers\init.pyi -> build\lib.win-amd64-cpython-312\tokenizers\pre_tokenizers copying py_src\tokenizers\processors\init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\processors copying py_src\tokenizers\trainers\init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\trainers copying py_src\tokenizers\tools\visualizer-styles.css -> build\lib.win-amd64-cpython-312\tokenizers\tools running build_ext running build_rust Updating crates.io index cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib -- Compiling autocfg v1.2.0 Compiling proc-macro2 v1.0.81 Compiling unicode-ident v1.0.12 Compiling windows_x86_64_msvc v0.52.5 Compiling cfg-if v1.0.0 Compiling jobserver v0.1.30 Compiling windows_x86_64_msvc v0.48.5 Compiling pkg-config v0.3.30 Compiling libc v0.2.153 Compiling version_check v0.9.4 Compiling typenum v1.17.0 Compiling memchr v2.7.2 Running rustc --crate-name autocfg "C:\Users\Daily Use\.cargo\registry\src\index.crates.io-6f17d22bba15001f\autocfg-1.2.0\src\lib.rs" --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=2c3fac6f07cbc8fa -C extra-filename=-2c3fac6f07cbc8fa --out-dir "C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps" -L "dependency=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps" --cap-lints allow Running rustc --crate-name build_script_build --edition=2021 "C:\Users\Daily Use\.cargo\registry\src\index.crates.io-6f17d22bba15001f\proc-macro2-1.0.81\build.rs" --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"proc-macro\"" -C metadata=33c3f25d26c45eb1 -C extra-filename=-33c3f25d26c45eb1 --out-dir "C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\build\proc-macro2-33c3f25d26c45eb1" -L "dependency=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps" --cap-lints allow Running rustc --crate-name unicode_ident --edition=2018 "C:\Users\Daily Use\.cargo\registry\src\index.crates.io-6f17d22bba15001f\unicode-ident-1.0.12\src\lib.rs" --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=ba2cd6e07fb41066 -C extra-filename=-ba2cd6e07fb41066 --out-dir "C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps" -L "dependency=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps" --cap-lints allow Running rustc --crate-name build_script_build --edition=2021 "C:\Users\Daily Use\.cargo\registry\src\index.crates.io-6f17d22bba15001f\windows_x86_64_msvc-0.52.5\build.rs" --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C metadata=89eacd1350eb4cfd -C extra-filename=-89eacd1350eb4cfd --out-dir "C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\build\windows_x86_64_msvc-89eacd1350eb4cfd" -L "dependency=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps" --cap-lints allow Running rustc --crate-name cfg_if --edition=2018 "C:\Users\Daily Use\.cargo\registry\src\index.crates.io-6f17d22bba15001f\cfg-if-1.0.0\src\lib.rs" --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=17892c9839bed1cb -C extra-filename=-17892c9839bed1cb --out-dir "C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps" -L "dependency=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps" --cap-lints allow

...

Running rustc --crate-name tokenizers --edition=2018 tokenizers-lib\src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"cached-path\"" --cfg "feature=\"clap\"" --cfg "feature=\"cli\"" --cfg "feature=\"default\"" --cfg "feature=\"dirs\"" --cfg "feature=\"esaxx_fast\"" --cfg "feature=\"http\"" --cfg "feature=\"indicatif\"" --cfg "feature=\"onig\"" --cfg "feature=\"progressbar\"" --cfg "feature=\"reqwest\"" -C metadata=7e0c7663af47560c -C extra-filename=-7e0c7663af47560c --out-dir "C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps" -L "dependency=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps" --extern "aho_corasick=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libaho_corasick-2e6901a7f66c9729.rmeta" --extern "cached_path=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libcached_path-9b7170c1f7920c9c.rmeta" --extern "clap=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libclap-7e39930c76eb894e.rmeta" --extern "derive_builder=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libderive_builder-710ea7cc442a8184.rmeta" --extern "dirs=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libdirs-71e9d29ab2461d90.rmeta" --extern "esaxx_rs=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libesaxx_rs-79fc94dcd6604592.rmeta" --extern "getrandom=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libgetrandom-04739bfd087cfcc1.rmeta" --extern "indicatif=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libindicatif-5d9c4120141ae88c.rmeta" --extern "itertools=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libitertools-94d15f64998aca8b.rmeta" --extern "lazy_static=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\liblazy_static-5f9444fdb29d44b7.rmeta" --extern "log=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\liblog-d84d806982d24cc6.rmeta" --extern "macro_rules_attribute=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libmacro_rules_attribute-eb9ad87d9eefd7cc.rmeta" --extern "monostate=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libmonostate-38fc1be511170de6.rmeta" --extern "onig=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libonig-4dee1b2ffee23683.rmeta" --extern "paste=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\paste-a29194707c74bce2.dll" --extern "rand=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\librand-d97f97db9460fea4.rmeta" --extern "rayon=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\librayon-eebc8b783646e934.rmeta" --extern "rayon_cond=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\librayon_cond-fef04d7b1eb6361d.rmeta" --extern "regex=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libregex-168971063707a96f.rmeta" --extern "regex_syntax=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libregex_syntax-cdbf23afbbe7ff91.rmeta" --extern "reqwest=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libreqwest-2d816465f8da4e20.rmeta" --extern "serde=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libserde-60a4df9e2653e944.rmeta" --extern "serde_json=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libserde_json-34aca260b202b64c.rmeta" --extern "spm_precompiled=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libspm_precompiled-4ecb4541aaa9a4ad.rmeta" --extern "thiserror=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libthiserror-1c4a897be38cf4e0.rmeta" --extern "unicode_normalization_alignments=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libunicode_normalization_alignments-3f364afba5614e5b.rmeta" --extern "unicode_segmentation=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libunicode_segmentation-bf2354f63ed11988.rmeta" --extern "unicode_categories=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libunicode_categories-93a00f926fbabe0c.rmeta" -L "native=C:\Users\Daily Use\.cargo\registry\src\index.crates.io-6f17d22bba15001f\windows_x86_64_msvc-0.52.5\lib" -L "native=C:\Users\Daily Use\.cargo\registry\src\index.crates.io-6f17d22bba15001f\windows_x86_64_msvc-0.48.5\lib" -L "native=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\build\bzip2-sys-cf42efa254fcae54\out\lib" -L "native=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\build\zstd-sys-4a283d60bc586915\out" -L "native=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\build\esaxx-rs-5ed8c9c732cbb7d7\out" -L "native=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\build\onig_sys-eab5e426da51abba\out" Compiling numpy v0.18.0 Running rustc --crate-name numpy --edition=2018 "C:\Users\Daily Use\.cargo\registry\src\index.crates.io-6f17d22bba15001f\numpy-0.18.0\src\lib.rs" --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=304e4dde5e5ad15d -C extra-filename=-304e4dde5e5ad15d --out-dir "C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps" -L "dependency=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps" --extern "libc=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\liblibc-bec58038b5a6abf9.rmeta" --extern "ndarray=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libndarray-a1d423cb09012739.rmeta" --extern "num_complex=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libnum_complex-8e28a50c6c9d2f58.rmeta" --extern "num_integer=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libnum_integer-8115cb4b58c688b8.rmeta" --extern "num_traits=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libnum_traits-5ff7168f2c8fc61d.rmeta" --extern "pyo3=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libpyo3-938cf004144be8f9.rmeta" --extern "rustc_hash=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\librustc_hash-bf77e6f8d503ddb0.rmeta" --cap-lints allow -L "native=C:\Users\Daily Use\.cargo\registry\src\index.crates.io-6f17d22bba15001f\windows_x86_64_msvc-0.48.5\lib" -L "native=C:\Users\Daily Use\AppData\Local\Programs\Python\Python312\libs" warning: variable does not need to be mutable --> tokenizers-lib\src\models\unigram\model.rs:265:21 265 let mut target_node = &mut best_path_ends_at[key_pos]; ----^^^^^^^^^^^
help: remove this mut
      = note: `#[warn(unused_mut)]` on by default

  warning: variable does not need to be mutable
     --> tokenizers-lib\src\models\unigram\model.rs:282:21
      |
  282 |                 let mut target_node = &mut best_path_ends_at[starts_at + mblen];
      |                     ----^^^^^^^^^^^
      |                     |
      |                     help: remove this `mut`

  warning: variable does not need to be mutable
     --> tokenizers-lib\src\pre_tokenizers\byte_level.rs:200:59
      |
  200 |     encoding.process_tokens_with_offsets_mut(|(i, (token, mut offsets))| {
      |                                                           ----^^^^^^^
      |                                                           |
      |                                                           help: remove this `mut`

  error: casting `&T` to `&mut T` is undefined behavior, even if the reference is unused, consider instead using an `UnsafeCell`
     --> tokenizers-lib\src\models\bpe\trainer.rs:526:47
      |
  522 |                     let w = &words[*i] as *const _ as *mut _;
      |                             -------------------------------- casting happend here
  ...
  526 |                         let word: &mut Word = &mut (*w);
      |                                               ^^^^^^^^^
      |
      = note: for more information, visit <https://doc.rust-lang.org/book/ch15-05-interior-mutability.html>
      = note: `#[deny(invalid_reference_casting)]` on by default

  warning: `tokenizers` (lib) generated 3 warnings
  error: could not compile `tokenizers` (lib) due to 1 previous error; 3 warnings emitted

  Caused by:
    process didn't exit successfully: `rustc --crate-name tokenizers --edition=2018 tokenizers-lib\src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"cached-path\"" --cfg "feature=\"clap\"" --cfg "feature=\"cli\"" --cfg "feature=\"default\"" --cfg "feature=\"dirs\"" --cfg "feature=\"esaxx_fast\"" --cfg "feature=\"http\"" --cfg "feature=\"indicatif\"" --cfg "feature=\"onig\"" --cfg "feature=\"progressbar\"" --cfg "feature=\"reqwest\"" -C metadata=7e0c7663af47560c -C extra-filename=-7e0c7663af47560c --out-dir "C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps" -L "dependency=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps" --extern "aho_corasick=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libaho_corasick-2e6901a7f66c9729.rmeta" --extern "cached_path=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libcached_path-9b7170c1f7920c9c.rmeta" --extern "clap=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libclap-7e39930c76eb894e.rmeta" --extern "derive_builder=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libderive_builder-710ea7cc442a8184.rmeta" --extern "dirs=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libdirs-71e9d29ab2461d90.rmeta" --extern "esaxx_rs=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libesaxx_rs-79fc94dcd6604592.rmeta" --extern "getrandom=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libgetrandom-04739bfd087cfcc1.rmeta" --extern "indicatif=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libindicatif-5d9c4120141ae88c.rmeta" --extern "itertools=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libitertools-94d15f64998aca8b.rmeta" --extern "lazy_static=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\liblazy_static-5f9444fdb29d44b7.rmeta" --extern "log=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\liblog-d84d806982d24cc6.rmeta" --extern "macro_rules_attribute=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libmacro_rules_attribute-eb9ad87d9eefd7cc.rmeta" --extern "monostate=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libmonostate-38fc1be511170de6.rmeta" --extern "onig=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libonig-4dee1b2ffee23683.rmeta" --extern "paste=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\paste-a29194707c74bce2.dll" --extern "rand=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\librand-d97f97db9460fea4.rmeta" --extern "rayon=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\librayon-eebc8b783646e934.rmeta" --extern "rayon_cond=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\librayon_cond-fef04d7b1eb6361d.rmeta" --extern "regex=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libregex-168971063707a96f.rmeta" --extern "regex_syntax=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libregex_syntax-cdbf23afbbe7ff91.rmeta" --extern "reqwest=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libreqwest-2d816465f8da4e20.rmeta" --extern "serde=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libserde-60a4df9e2653e944.rmeta" --extern "serde_json=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libserde_json-34aca260b202b64c.rmeta" --extern "spm_precompiled=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libspm_precompiled-4ecb4541aaa9a4ad.rmeta" --extern "thiserror=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libthiserror-1c4a897be38cf4e0.rmeta" --extern "unicode_normalization_alignments=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libunicode_normalization_alignments-3f364afba5614e5b.rmeta" --extern "unicode_segmentation=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libunicode_segmentation-bf2354f63ed11988.rmeta" --extern "unicode_categories=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\deps\libunicode_categories-93a00f926fbabe0c.rmeta" -L "native=C:\Users\Daily Use\.cargo\registry\src\index.crates.io-6f17d22bba15001f\windows_x86_64_msvc-0.52.5\lib" -L "native=C:\Users\Daily Use\.cargo\registry\src\index.crates.io-6f17d22bba15001f\windows_x86_64_msvc-0.48.5\lib" -L "native=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\build\bzip2-sys-cf42efa254fcae54\out\lib" -L "native=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\build\zstd-sys-4a283d60bc586915\out" -L "native=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\build\esaxx-rs-5ed8c9c732cbb7d7\out" -L "native=C:\Users\Daily Use\AppData\Local\Temp\pip-install-zr2i9v5l\tokenizers_4f60e45c2dcc4314bfbeb3f47233aa35\target\release\build\onig_sys-eab5e426da51abba\out"` (exit code: 1)
  warning: build failed, waiting for other jobs to finish...
  error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib --` failed with code 101
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for tokenizers Failed to build xformers tokenizers ERROR: Could not build wheels for xformers, tokenizers, which is required to install pyproject.toml-based projects

C:\Users\Daily Use\Documents\Lora Training\ComfyUI_windows_portable\ComfyUI>

yasushisakai commented 5 months ago

linking this upstream issue: https://github.com/huggingface/tokenizers/issues/1485 tokenizers seem to have trouble compiling with the latest version of rust. Other parts of tokenizer code depend on newer versions of rust so ATM its deadlocked.

okolenmi commented 4 months ago

Tokenizers are fixed a long time ago but transformers wants old 0.11.3 version of tokenizers.