issues
search
runpod-workers
/
worker-vllm
The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
MIT License
213
stars
82
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
weird output when using a custom model and ChatAPI does not work
#55
Mr-Nobody1
closed
5 months ago
7
Support for GPT3 based models
#54
letajmal
closed
6 months ago
1
MODEL_REVISION not read
#53
Sapessii
closed
6 months ago
2
[WIP] Testing Suite
#52
alpayariyak
closed
3 weeks ago
0
Do the new images work?
#51
dannysemi
closed
6 months ago
13
feat: auto build cuda version
#50
justinmerrell
closed
6 months ago
0
Cannot run Mixtral 8x7B Instruct AWQ
#49
ddemillard
closed
6 months ago
6
Fixes import statement
#48
rachfop
closed
6 months ago
0
v0.3.0: OpenAI Compatibility, Dynamic Stream Batching, Refactor, Error Responses, more
#47
alpayariyak
closed
6 months ago
0
Huggingface is down and my worker is looping
#46
dannysemi
closed
6 months ago
1
fix: build error if no `TOKENIZER_NAME` provided
#45
willsamu
closed
6 months ago
1
Update runpod package version
#44
github-actions[bot]
closed
6 months ago
0
trust_remote_code not recognized
#43
dannysemi
closed
6 months ago
1
Error after tokenizer commit
#42
StableFluffy
closed
6 months ago
1
Support for mistralai/Mixtral-8x7B-Instruct-v0.1
#41
ilkersigirci
closed
6 months ago
1
enforce_eager flag
#40
dannysemi
closed
6 months ago
5
Download additional files for build
#39
casper-hansen
closed
6 months ago
0
Update runpod package version
#38
github-actions[bot]
closed
6 months ago
0
Docker image is taking too much time to build
#37
hiennef
closed
7 months ago
7
`MAX_CONCURRENCY` parameter doesn't work
#36
antonioglass
closed
7 months ago
1
feat: add `max_model_length` setup key
#35
willsamu
closed
7 months ago
2
Add TORCH_CUDA_ARCH_LIST to Dockerfile
#34
bartlettD
closed
7 months ago
4
"n" parameter does not return multiple responses
#33
hexadecible
closed
7 months ago
1
python setup.py develop did not run successfully
#32
heraistudios
closed
7 months ago
3
test_input.json is required
#31
wwydmanski
closed
7 months ago
2
Update runpod package version
#30
github-actions[bot]
closed
7 months ago
0
Errors cause the instance to run indefinitely
#29
gabewillen
opened
8 months ago
23
Update Dockerfile env key that is deprecated
#28
nerdylive123
closed
7 months ago
1
Chat Template Feature, Message List, Small Refactor
#27
alpayariyak
closed
8 months ago
0
Build not possilbe
#26
saaldigital-cw
closed
8 months ago
2
Errors when building the image [Building on MACOS]
#25
antonioglass
closed
7 months ago
2
Runpod
#24
saaldigital-cw
closed
8 months ago
4
Default parameters for MAX_CONCURRENCY and DEFAULT_BATCH_SIZE are reversed in utils.py
#23
nicolasembleton
closed
8 months ago
2
Sampling parameter "stop" doesn't work with the new worker-vllm
#22
antonioglass
closed
8 months ago
1
HF Model Download get stuck
#21
satpalsr
closed
8 months ago
7
fix: update badge
#20
justinmerrell
closed
8 months ago
0
Update runpod package version
#19
github-actions[bot]
closed
8 months ago
0
10x Faster New Worker
#18
alpayariyak
closed
8 months ago
0
Set TOKENIZER to None instead of empty string if not provided
#17
ashleykleynhans
closed
8 months ago
1
Feat: AWQ quantisation support
#16
willsamu
closed
9 months ago
2
PSA: Here's how to get it working with Mistral
#15
Palmik
closed
9 months ago
1
PSA: Things break if you don't set TOKENIZER variable
#14
Palmik
closed
8 months ago
3
PSA: Streaming is slow
#13
Palmik
closed
8 months ago
3
support for quantization?
#12
WillReynolds5
closed
9 months ago
3
Update vllm version to 0.2.1
#11
kenny019
closed
9 months ago
1
Support custom vLLM build
#10
viktor-ferenczi
closed
8 months ago
2
Update for vllm 0.2.0
#9
kenny019
closed
9 months ago
0
Could not build wheels for vllm
#8
Mr-Nobody1
closed
8 months ago
1
modified with sysprompt support
#7
Reverendissimo
closed
7 months ago
0
Rev llama worker runpod
#6
Reverendissimo
closed
7 months ago
2
Previous
Next