issues
search
scaleapi
/
llm-engine
Scale LLM Engine public repository
https://llm-engine.scale.com
Apache License 2.0
769
stars
50
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Chat completion API
#609
dmchoiboi
opened
2 days ago
0
Http Forwarder updates
#608
dmchoiboi
closed
1 day ago
1
vllm_batch: Fix list initialization
#607
dmchoiboi
closed
1 week ago
0
Relax pydantic constraint for client
#606
dmchoiboi
closed
2 weeks ago
4
More batch job cleanup
#605
dmchoiboi
closed
2 weeks ago
0
Some cleanups for batch v2
#604
dmchoiboi
closed
2 weeks ago
0
MLI-2847 Replace instead of patch PDB
#603
yunfeng-scale
closed
1 week ago
0
Batch completions V2 job
#602
dmchoiboi
closed
2 weeks ago
2
Docs for qwen2 72b instruct
#601
yunfeng-scale
closed
1 week ago
0
Update client with new max_context_length
#600
dmchoiboi
closed
3 weeks ago
0
Fix dto for batch completion
#599
dmchoiboi
closed
3 weeks ago
0
Allow setting max context length for batch jobs
#598
dmchoiboi
closed
3 weeks ago
0
Get v2 batch inference working
#597
seanshi-scale
closed
1 day ago
0
Use maxUnavailale for endpoint PDB
#596
yunfeng-scale
closed
1 week ago
0
Get Gemma2 working
#595
dmchoiboi
closed
3 weeks ago
1
Add Qwen2 72b instruct
#594
yunfeng-scale
closed
3 weeks ago
3
[Batch Completions V2] DTO models + Batch completions service
#593
dmchoiboi
closed
3 weeks ago
0
MLI-2510 Validate json logs to test hypothesis on no records in Snowflake
#592
tiffzhao5
closed
3 weeks ago
0
Allow support for vllm batch with checkpoints
#591
dmchoiboi
closed
3 weeks ago
0
More Batch Inference Options
#590
seanshi-scale
closed
4 weeks ago
0
Multinode batch inference
#589
seanshi-scale
opened
1 month ago
0
Re-add auth header
#588
dmchoiboi
closed
1 month ago
0
Make storage required for endpoint creation requests
#587
squeakymouse
closed
1 month ago
0
Enable passing in headers through the client
#586
dmchoiboi
closed
1 month ago
0
Various Db configuration improvements
#585
dmchoiboi
closed
1 month ago
0
Add env label
#584
yunfeng-scale
closed
1 month ago
0
Use session for sts boto3 client for logging hook
#583
tiffzhao5
closed
1 month ago
0
Reduce connection pool size
#582
yunfeng-scale
closed
1 month ago
0
Make configs backwards-compatible
#581
squeakymouse
closed
1 month ago
1
Bump istio proxy memory for gateway
#580
yunfeng-scale
closed
1 month ago
0
Add autogenerated openai spec
#579
dmchoiboi
closed
1 month ago
2
Shared pydantic configs
#578
dmchoiboi
closed
1 month ago
0
Add Llama 3.1 models
#577
seanshi-scale
closed
1 month ago
0
Bump vllm version to 0.5.3post1
#576
seanshi-scale
closed
1 month ago
2
Up storage limit
#575
dmchoiboi
closed
1 month ago
0
Multinode serving
#574
seanshi-scale
opened
1 month ago
0
Reduce hardware requirement for deepseek coder lite
#573
dmchoiboi
closed
1 month ago
0
Add deepseek models
#572
dmchoiboi
closed
1 month ago
0
Bump vllm batch version
#571
dmchoiboi
closed
1 month ago
1
Disable data parallelism for batch completions
#570
dmchoiboi
closed
1 month ago
0
Log info instead of debug
#569
yunfeng-scale
closed
1 month ago
0
Add earliest log
#568
yunfeng-scale
closed
1 month ago
1
Allow request deserialization using alias
#567
dmchoiboi
closed
1 month ago
0
Parse wrapped sync endpoint error
#566
yunfeng-scale
closed
1 month ago
0
Bump setuptools from 69.0.3 to 70.0.0 in /model-engine
#565
dependabot[bot]
opened
1 month ago
0
Add support for phi 3 models
#564
dmchoiboi
closed
1 month ago
0
fix: Use env AWS_REGION in sqs_client or default to us-west-2
#563
nicolastomeo
closed
1 month ago
0
fix some oddities in the client
#562
seanshi-scale
closed
1 month ago
0
Bump pydantic to 2.8.2
#561
dmchoiboi
closed
1 month ago
0
Update vllm server to be openai compatible
#560
dmchoiboi
closed
1 month ago
0
Next