-
AWSGH updated
2 weeks ago
-
Hi,
I'm trying to trace the vision encoder part of Meta's Segment Anything Model (SAM), and I'm encountering several errors during the trace process but it seems to be stuck now.
The script does…
-
I have inf2.24xlarge and I am running the Llama-2 inference example. All the packages are installed latest.
Everything worked fine until the step where I load model with tp_degree = 24 and it faile…
-
We are trying to run the [e5 model](https://huggingface.co/intfloat/e5-large-v2) on an inf2 instance. The model compiles fine and analyze reports no unsupported operators but when trying it out on an …
aabbi updated
2 weeks ago
-
## 論文タイトル（原文まま）
Reliability-Neurons: Investigating Neurons that Predict Model Uncertainty
## 一言でいうと
大規模言語モデル（LLM）の不確実性を調整するエントロピーニューロンとトークン頻度ニューロンという2つの主要なコンポーネントを特定し、そのメカニズムを解明した研究。
### 論文リンク…
-
Error when running sample **python3 examples/offline_inference_neuron.py**, after installing v0.3.3 (from cloned source or from pip install git+...).
### Cause:
directory **vllm/model_executor/m…
-
Hi,
We (@zml) found that `tf.aliasing` support seemed to be not working as expected, with the model producing garbage when used. In our case Llama 3.1 8B.
This is problematic for transformer model…
-
**Describe the bug**
When using `InhomogeneousParameter` in an `all` segment group, the generated NEURON does not run.
**To Reproduce**
Add an InhomogeneousParameter to an `all` segment group.
…
-
I am getting segmentation fault when trying to compile model jinaai/jina-embeddings-v2-base-en with neuronx.
Example script:
```python
from transformers import AutoTokenizer, AutoModel
import …
-
I am trying to use the Allocation API to manually allocate tensors in my NKI kernel. Unfortunately, even with a simple kernel that exponentiates every element, I am seeing an error. I have confirmed t…