issues
search
substratusai
/
lingo
Lightweight ML model proxy and autoscaler for kubernetes
https://www.substratus.ai
Apache License 2.0
102
stars
6
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Dedup received messages
#118
nstogner
opened
2 weeks ago
0
remove a sleep that's not needed
#117
samos123
closed
2 weeks ago
0
Recreate subscription on receive message error and add pubsub e2e test
#116
samos123
closed
2 weeks ago
0
Revert "Fix lingo crash due to EXACTLY_ONCE_ACKID_FAILURE (#112)"
#115
samos123
closed
2 weeks ago
0
fix autoscaler stats aggregation
#114
samos123
closed
2 weeks ago
0
Load balancing doesn't seem to spread evenly
#113
samos123
opened
2 weeks ago
4
Fix lingo crash due to EXACTLY_ONCE_ACKID_FAILURE
#112
samos123
closed
2 weeks ago
0
Configuration options for hardcoded values
#111
nstogner
closed
3 weeks ago
4
Add test cases for movingaverage
#110
nstogner
closed
2 weeks ago
0
Messenger integration: Make error backoff configurable
#109
nstogner
closed
3 weeks ago
0
Feature request: ability to configure the time window used to calculate the average active requests.
#108
nstogner
closed
3 weeks ago
0
fix #106 improve scale down behavior
#107
samos123
closed
2 weeks ago
5
Improve scaling behavior when there are requests waiting to be queued
#106
samos123
closed
2 weeks ago
1
health check results in unable to parse model error
#105
samos123
opened
3 weeks ago
1
add script to create sa and copy to clipboard
#104
samos123
closed
3 weeks ago
0
Add docs on how to use pub/sub integration
#103
samos123
opened
1 month ago
0
add end to end private RAG example
#102
samos123
closed
2 months ago
0
Support OpenAI API key based authentication
#101
samos123
opened
3 months ago
0
lingo messenger crashes causes restart of lingo
#100
samos123
opened
3 months ago
2
add flash attention in vLLM helm chart
#99
samos123
closed
3 months ago
0
e2e messenger GCP pubsub system tests
#98
samos123
opened
3 months ago
0
Add hostname to message metadata
#97
nstogner
opened
3 months ago
0
vLLM ocasionally gets into broken state
#96
samos123
opened
3 months ago
0
Restarting lingo should not cause instant scale downs
#95
samos123
opened
3 months ago
0
Fix missing return - causing duplicate calls to sendResponse - triggering a panic
#94
nstogner
closed
3 months ago
0
Messenger: Panic panic: Ack/Nack called twice on
#93
samos123
closed
3 months ago
0
Messenger: Log the metadata of each message
#92
samos123
opened
3 months ago
0
messenger: improve concurrent handling
#91
samos123
closed
3 months ago
0
Bucket integration
#90
nstogner
opened
3 months ago
0
Retry failures when consuming requests via messaging integration
#89
nstogner
opened
3 months ago
0
Messaging integration (GCP PubSub, AWS SQS, Kafka, etc)
#88
nstogner
closed
3 months ago
12
lingo ha mode failed to aggregate stats
#87
samos123
opened
4 months ago
0
Batch support through Pub/Sub
#86
samos123
closed
3 months ago
8
Expose vLLM metrics through lingo
#85
samos123
opened
4 months ago
0
CNCF TAG-Runtime or/and CNCF Cloud Native WG discussion
#84
raravena80
closed
4 months ago
2
[Spike] Introduce deployment annotation
#83
alpe
opened
5 months ago
0
remove need to name deployment same as service
#82
samos123
opened
5 months ago
0
GitHub action for creating release
#81
samos123
closed
3 months ago
3
Spike: Reconcile pods for model ip addresses
#80
alpe
opened
5 months ago
0
Refactor logging to use library
#79
samos123
opened
5 months ago
2
fix missing test-race in make test
#78
samos123
closed
5 months ago
1
Only get stats and autoscale deployments with model annotation
#77
samos123
opened
5 months ago
0
Fix scale back to 0 with 300 requests scenario
#76
samos123
closed
5 months ago
0
[Do not merge] alternative approach to clear state when not leader anymore
#75
alpe
closed
5 months ago
1
[Do not merge]scaler reset desired scale
#74
alpe
closed
5 months ago
0
Scale to 0 not working with replicas 3
#73
samos123
closed
5 months ago
0
Fix #67 only leader should do scaling
#72
samos123
closed
5 months ago
1
Do not merge: #70 on top of #66
#71
alpe
closed
5 months ago
1
Stop scale down timer
#70
alpe
closed
5 months ago
2
Spike: Integration test fix
#69
alpe
closed
5 months ago
2
Next