-
### Your current environment
```text
The output of `python collect_env.py`
```
### How would you like to use vllm
I want to run inference of a llama3. I don't know how to integrate it with vllm…
-
This is a meta-issue to serve as a list of the different integrations between metatensor atomistic models and simulation engines. Any longer discussion should be split into a separate issue and linked…
-
A sound similar to electrical buzzing
I tried sideloading several browsers and found that those based on the Chrome engine experienced this issue, while those powered by the Firefox engine worked fin…
-
### Description
Bedrock engine streaming is currently broken. Should affect all model types in bedrock, but need to test Titan, Meta, Claude, etc.
### Existing
Is there any relevant code or…
-
Hi
I wanted to have a model using NVIDIA H100 GPU instead of NVIDIA T4 GPU.
In the values possible I see:
```
llama-3.1-8b-instruct-fp8-l4:
enabled: false
features: ["TextGeneration"]
…
-
## Current Behavior
When vacuuming a table, the following sequence occurs:
1. Data is removed from S3.
2. Table metadata is removed from the meta-service.
However, the current API allows undro…
-
**Describe the bug**
Formatting high numbers like "1000 → 1k", "1000000 → 1M" should be localized, and "k", "m" chars shouldn't be hardcoded.
**Note for the developer:**
https://github.com/wooc…
-
**What happened**:
https://github.com/juicedata/juicefs/actions/runs/9057923627/job/24882685721
```
============================= test session starts ==============================
platform linu…
-
When I run _Meta-Llama-3-8B-Instruct_ or _Meta-Llama-3.1-8B-Instruct_ with
1. python 3.12.5
2. scalellm 0.1.9+cu118torch2.2.2
3. torch 2.2.2+cu1…
-
### Prerequisites
- [ ] I have carried out troubleshooting steps and I believe I have found a bug.
- [X] I have searched for similar bugs in both open and closed issues and cannot find a duplicate.
…