-
My conclusions about changing the positional encoding are that NOPE and ALiBi do not work well for only-encoders because, compared to only-decoders, they do not understand position at all (they are …
-
**Describe the bug**
If using `--reset-position-ids`, the RoPE implementation does not take this into account; it will still use the embeddings from position 0 to sequence length - 1.
**To Reprodu…
-
### Model description
jina-embeddings-v3 is a multilingual multi-task text embedding model designed for a variety of NLP applications. Based on the [Jina-XLM-RoBERTa architecture](https://huggingface…
-
My model is
```json
{
"mlp_bias": false,
"attn_bias": false,
"rotary_base": 300000,
"rotary_scaling": null,
"residual_mlp": false,
"disable_weight_only_quant_plugin": false,
…
-
I'm testing unsloth rope and here is my script:
```python
import torch
from unsloth.kernels.rope_embedding import fast_rope_embedding
from unsloth.models.llama import LlamaRotaryEmbedding as Uns…
-
Today, ROPE in decode mode is implemented as a matmul, where rot_mat is precomputed on host based on the sin/cos for each user's position_id.
What we want is for the `sin,cos: [max_seq_len=128k, he…
-
System Info
CPU architecture ( x86_64)
CPU/Host memory size (64GB)
GPU properties
GPU name ( NVIDIA RTX4090)
GPU memory size (24GB)
Libraries
TensorRT-LLM branch or tag (v0.13.0)
Versions of Tenso…
-
jina-embeddings-v3 is a multilingual multi-task text embedding model designed for a variety of NLP applications. Based on the [Jina-XLM-RoBERTa architecture](https://huggingface.co/jinaai/xlm-roberta-…
-
Trying to have Open Interpreter describe images locally. Errors every time. I use a Mac with Apple silicon
Not sure if the issue is with how Open Interpreter is passing images to moondream [file](h…
-
### Your current environment
```text
The output of `python collect_env.py`
```
### 🐛 Describe the bug
There is a difference in the vLLM implementation of DynamicNTKScalingRotaryEmbedding from t…