Open starstream opened 1 month ago
I change Q = Q.view(batchseq_len, n_headshead_dim) --> Q = Q.reshape(batchseq_len, n_headshead_dim)
beacuse of error:
File "/opt/conda/lib/python3.10/site-packages/unsloth/kernels/rope_embedding.py", line 79, in forward
Q = Q.view(batchseq_len, n_headshead_dim)
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.
Is that the reason? but I think the two approaches are equivalent.
It should be equivalent - the issue is
cos = torch.rand(seq_length, head_dims, device=0)
sin = torch.rand(seq_length, head_dims, device=0)
is not actually correct - the cos and sin matrices are actually duplicated, hence you'll get different numbers if you naively call them.
One way to check them is to manually edit Unsloth's code during the application of the RoPE kernel, and call the slow and fast methods together, then use torch.dist
and print the difference out
Is fast rope exactly equivalent to llama's apply_rotary_pos_emb? I constructed a test case and found that the result is not exactly equivalent. Is there anything wrong with my case
code:
output: