FurkanGozukara
commented
1 month ago it is super hard to self compile i hope pre compiled wheels get published for python 3.10 and 3.11
Closed osadchi closed 1 month ago
FurkanGozukara
commented
1 month ago it is super hard to self compile i hope pre compiled wheels get published for python 3.10 and 3.11
osadchi
commented
1 month ago it is super hard to self compile i hope pre compiled wheels get published for python 3.10 and 3.11
But there's 2.0.0 and 2.1.0 is this too old?
FurkanGozukara
commented
1 month ago it is super hard to self compile i hope pre compiled wheels get published for python 3.10 and 3.11
But there's 2.0.0 and 2.1.0 is this too old?
yes we need 3.0
osadchi
commented
1 month ago it is super hard to self compile i hope pre compiled wheels get published for python 3.10 and 3.11
But there's 2.0.0 and 2.1.0 is this too old?
yes we need 3.0
I think I'm on the right way, but cant clearly see what's the point of the error. May be its llvm incompatible with current triton :C
osadchi
commented
1 month ago wsl> > > it is super hard to self compile i hope pre compiled wheels get published for python 3.10 and 3.11
But there's 2.0.0 and 2.1.0 is this too old?
yes we need 3.0
https://github.com/jakaline-dev/Triton_win/releases/tag/3.0.0
Did you try this ones? I have an error with this module
from triton.language.extra import libdevice
ImportError: cannot import name 'libdevice' from 'triton.language.extra'After all so far, I can successfully run llamafactory It loads the model and can run good. But I still cant fine-tune it
Getting this error from jit it calls wrong attribute?!
10/17/2024 02:15:19 - INFO - llamafactory.model.model_utils.checkpointing - Gradient checkpointing enabled.
10/17/2024 02:15:19 - INFO - llamafactory.model.adapter - Upcasting trainable params to float32.
10/17/2024 02:15:19 - INFO - llamafactory.model.adapter - Fine-tuning method: LoRA
10/17/2024 02:15:19 - INFO - llamafactory.model.model_utils.misc - Found linear modules: o_proj,down_proj,up_proj,k_proj,q_proj,gate_proj,v_proj
[WARNING|logging.py:328] 2024-10-17 02:15:21,142 >> Unsloth 2024.7 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers.
10/17/2024 02:15:22 - INFO - llamafactory.model.loader - trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605
[INFO|trainer.py:642] 2024-10-17 02:15:22,197 >> Using auto half precision backend
[WARNING|<string>:223] 2024-10-17 02:15:22,472 >> ==((====))== Unsloth - 2x faster free finetuning | Num GPUs = 1
\\ /| Num examples = 1 | Num Epochs = 1
O^O/ \_/ \ Batch size per device = 1 | Gradient Accumulation steps = 8
\ / Total batch size = 8 | Total steps = 1
"-____-" Number of trainable parameters = 20,971,520
0%| | 0/1 [00:00<?, ?it/s]ptxas info : 11 bytes gmem
ptxas info : Compiling entry function '_rms_layernorm_forward' for 'sm_86'
ptxas info : Function properties for _rms_layernorm_forward
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 40 registers, 408 bytes cmem[0]
ptxas info : 0 bytes gmem
ptxas info : Compiling entry function '_fg_kernel' for 'sm_86'
ptxas info : Function properties for _fg_kernel
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 30 registers, 380 bytes cmem[0]
Traceback (most recent call last):
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\ProgramData\anaconda3\envs\unsloth_env\Scripts\llamafactory-cli.exe\__main__.py", line 7, in <module>
sys.exit(main())
File "C:\LLaMA-Factory\src\llamafactory\cli.py", line 111, in main
run_exp()
File "C:\LLaMA-Factory\src\llamafactory\train\tuner.py", line 50, in run_exp
run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)
File "C:\LLaMA-Factory\src\llamafactory\train\sft\workflow.py", line 96, in run_sft
train_result = trainer.train(resume_from_checkpoint=training_args.resume_from_checkpoint)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\transformers\trainer.py", line 1932, in train
return inner_training_loop(
File "<string>", line 367, in _fast_inner_training_loop
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\transformers\trainer.py", line 3307, in training_step
loss = self.compute_loss(model, inputs)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\transformers\trainer.py", line 3338, in compute_loss
outputs = model(**inputs)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\accelerate\utils\operations.py", line 820, in forward
return model_forward(*args, **kwargs)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\accelerate\utils\operations.py", line 808, in __call__
return convert_to_fp32(self.model_forward(*args, **kwargs))
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\torch\amp\autocast_mode.py", line 16, in decorate_autocast
return func(*args, **kwargs)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\unsloth\models\llama.py", line 930, in PeftModelForCausalLM_fast_forward
return self.base_model(
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\peft\tuners\tuners_utils.py", line 179, in forward
return self.model.forward(*args, **kwargs)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\unsloth\models\llama.py", line 890, in _CausalLM_fast_forward
loss = fast_cross_entropy_loss(
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\unsloth\kernels\cross_entropy_loss.py", line 317, in fast_cross_entropy_loss
loss = Fast_CrossEntropyLoss.apply(
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\torch\autograd\function.py", line 598, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\unsloth\kernels\cross_entropy_loss.py", line 256, in forward
_chunked_cross_entropy_forward[(n_rows, n_chunks,)](
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\triton\runtime\jit.py", line 345, in <lambda>
return lambda *args, **kwargs: self.run(grid=grid, warmup=False, *args, **kwargs)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\triton\runtime\autotuner.py", line 338, in run
return self.fn.run(*args, **kwargs)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\triton\runtime\jit.py", line 662, in run
kernel = self.compile(
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\triton\compiler\compiler.py", line 244, in compile
key = f"{triton_key()}-{src.hash()}-{backend.hash()}-{options.hash()}-{str(sorted(env_vars.items()))}"
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\triton\compiler\compiler.py", line 109, in hash
key = f"{self.fn.cache_key}-{self.attrs.hash()}-{sorted_sig}-{sorted_constants}"
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\triton\runtime\jit.py", line 758, in cache_key
dependencies_finder.visit(self.parse())
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\ast.py", line 418, in visit
return visitor(node)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\ast.py", line 426, in generic_visit
self.visit(item)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\ast.py", line 418, in visit
return visitor(node)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\triton\runtime\jit.py", line 167, in visit_FunctionDef
self.generic_visit(node)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\ast.py", line 426, in generic_visit
self.visit(item)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\ast.py", line 418, in visit
return visitor(node)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\ast.py", line 426, in generic_visit
self.visit(item)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\ast.py", line 418, in visit
return visitor(node)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\triton\runtime\jit.py", line 221, in visit_Assign
self.generic_visit(node)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\ast.py", line 428, in generic_visit
self.visit(value)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\ast.py", line 418, in visit
return visitor(node)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\ast.py", line 428, in generic_visit
self.visit(value)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\ast.py", line 418, in visit
return visitor(node)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\triton\runtime\jit.py", line 127, in visit_Call
func = self.visit(node.func)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\ast.py", line 418, in visit
return visitor(node)
File "C:\ProgramData\anaconda3\envs\unsloth_env\lib\site-packages\triton\runtime\jit.py", line 117, in visit_Attribute
return getattr(lhs, node.attr)
AttributeError: module 'triton.language.math' has no attribute 'tanh'
woct0rdho
commented
1 month ago triton.language.math has functions like sin and cos, but no tanh yet. You can ask in the official triton repo
Thank you for your effort, I'm trying to build with your guide, I've tried both 3.0.0 and the 'main' branch. Both the same error. Llvm getting with hash in repo. https://github.com/llvm/llvm-project/tree/61f8a7f618901797ee8663389a29722f29216a96 this is my environment