-
## Overview
This issue tracks the support of RoPE scaling, one important configurable parameter adopted by many new models, in MLC LLM.
## Action Items
- [ ] Support linear RoPE scaling…
-
Hello, a project of mine requires a rope constraint that is updating it's length during run, I tried to use winch for this but due to winches custom physics that make it super unrealistic on its behav…
-
In full rope example, we have a fusion starting with something like this:
```
Inputs:
T0_g[ iS467{1024}, iS468{8} ], __bfloat
T1_g[ iS471{1024}, iS472{8} ], __bfloat
T2_g[ iS463{2}, iS464…
-
Hi @lucidrains,
These folks talk about improving axial-RoPE performance. Some comparisons to axial-RoPE look nice, but for some, I am not convinced. I wanted to get your thoughts on this. If it mak…
-
Hi, it seems that unsloth currently does not support loading base model trained by [OLMo](https://github.com/allenai/OLMo). Is it possible to write custom script to load the model into unsloth? The mo…
-
Hi, @lucidrains !
There was a promising research published this month (vs. RoPE-mixed (#25) in March), the so-called LieRE positional encodings generalize the kv-vector rotation to any numbers of d…
-
Hi, do you remember the type of rope and how many leds/m you have?? your link for it is dead.
Thanks
-
**Issue: Model Error when Setting max_seq_length > 8192**
**Description:**
The `unsloth/codegemma-2b-bnb-4bit` model throws an error when attempting to set `max_seq_length` greater than 8192.
…
-
![image](https://github.com/flashinfer-ai/flashinfer/assets/32770237/e13571da-9bff-456b-b67b-764aee66b7fd)
I found that during shared-prefix calculation, this kenerl won't use _qo_indptr_ to split ba…
-
When using a server, one currently cannot use the `model_overide_args` which could be very useful, e.g. for rope scaling.
This is currently the `sglang.launch_server.py`:
```py
import argparse…