-
### Describe the bug
With a fresh install of 1.15, Exllamav2_HF loads a model just fine... However, when I do a local install of exllamav2, then both it and the Exllamav2_HF loaders break ( errors b…
-
### Problem:
**400 Error message** With Vertex BatchPrediction API with _gemini-1.5-flash-001_ and _gemini-1.5-pro-001_
I am getting similar issues using Batch Prediction with gemini-1.5-flash-001…
-
We'd like to see the following application naming options:
- Default (Prefix (user defined), App Name, Suffix (environment))
- Options to disable prefix and suffix (2 separate options)
- Option …
-
-
### Your current environment
- vLLM version: v0.5.3.post1 (Public Docker Image )
- Model: Llama 3 70 B
- Dtype: FP16
- GPU: Nvidia H100
### 🐛 Describe the bug
The vLLM metrics endpoint is showin…
-
Is it possible to customise how the names of generic references are generated?
If I have for example the following:
```go
type Foo int
type Bar int
type FooMap[T any] map[Foo]T
type …
-
```
cd /root/workspace/github/optimum-habana/examples/text-generation/
python run_generation.py \
--model_name_or_path /root/workspace/model/meta-llama/Llama-3.1-8B/ \
--use_hpu_graphs \
--use_kv…
-
### System Info
```
2024-11-06T04:38:58.950145Z INFO text_generation_launcher: Runtime environment:
Target: x86_64-unknown-linux-gnu
Cargo version: 1.80.1
Commit sha: b1f9044d6cf082423a517cf9a6a…
-
### Skill Name
vike
### Why?
Next Generation Frontend Framework
- Next.js/Nuxt alternative
- Flexible
- Reliable
- Fast
- Clutter-free
- Community-driven
### Reference Image
h…
-
### Summary
We recently had out entire graphQL failover on an instance in Heartcore and it was due to using the reserved word "element" as an element type. Can you implement a non-allow or warning ba…