issues
search
Lightning-AI
/
LitServe
Deploy AI models at scale. High-throughput serving engine for AI/ML models that uses the latest state-of-the-art model deployment techniques.
https://lightning.ai/docs/litserve
Apache License 2.0
105
stars
9
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Track device automatically
#123
aniketmaurya
closed
15 hours ago
1
Track device automatically.
#122
williamFalcon
closed
15 hours ago
0
during manual local testing, the processes are not killed if the test fails
#121
aniketmaurya
opened
22 hours ago
0
Question: Serving LLama3-70B on multiple GPUs
#120
BZandi
opened
1 day ago
4
feat: Update OpenAI spec to include support for tools
#119
bhimrazy
closed
20 hours ago
7
inject context to share/log values across LitAPI methods
#118
aniketmaurya
closed
17 hours ago
5
Feature: Add support to include tools for OpenAI Chat Template to enable function calling.
#117
bhimrazy
closed
20 hours ago
3
failed to start and serve HTTP while server is in intermediate state i.e. shutting down
#116
gulshansainis
opened
6 days ago
5
% character when streaming
#115
rasbt
closed
1 week ago
2
Add real docs!
#114
williamFalcon
closed
1 week ago
1
feat: Update OpenAI spec to include image url in message content
#113
bhimrazy
closed
1 week ago
14
fix:Adds import statement for ChatMessage in OpenAI compatible API example
#112
bhimrazy
closed
1 week ago
0
Missing `ChatMessage` import in OpenAI compatible API in Readme
#111
bhimrazy
closed
1 week ago
0
Feat: Add support for fastapi lifespan events
#110
robTheBuildr
opened
1 week ago
3
investigate recycling pipes
#109
aniketmaurya
opened
1 week ago
0
fix pipe reuse
#108
aniketmaurya
closed
1 week ago
1
Add support to include images for OpenAI Chat Template
#107
bhimrazy
closed
1 week ago
3
concurrent API requests for OpenAI Spec results in inconsistent results
#106
aniketmaurya
closed
1 week ago
1
add newline while streaming JSON objects
#105
aniketmaurya
closed
1 week ago
1
add a newline while streaming JSON objects
#104
aniketmaurya
closed
1 week ago
0
[WIP] dynamic stream loop selection
#103
aniketmaurya
opened
1 week ago
0
Fix installation instructions for case sensitive file systems
#102
rasbt
closed
1 week ago
0
add openai streaming
#101
aniketmaurya
closed
1 week ago
3
Improve docs for OpenAI spec
#100
lantiga
closed
1 week ago
1
dynamic inference worker
#99
aniketmaurya
opened
1 week ago
0
add client detail to streaming document
#98
aniketmaurya
closed
1 week ago
1
fix summary tag
#97
aniketmaurya
closed
1 week ago
1
Exclude markdown files from pre-commit formatter
#96
rasbt
closed
1 week ago
1
Add Stream API Example
#95
rasbt
closed
1 week ago
1
Bump version since litserve is different from the last released version
#94
rasbt
closed
1 week ago
0
removes `/stream-predict` endpoint and standardizes to `/predict`
#93
aniketmaurya
closed
2 weeks ago
1
validate port
#92
aniketmaurya
closed
2 weeks ago
1
add custom endpoint path
#91
aniketmaurya
closed
2 weeks ago
2
enable users to override the default api endpoint path
#90
aniketmaurya
closed
2 weeks ago
0
Update server.py
#89
williamFalcon
closed
2 weeks ago
1
Too many files open with 1K requests
#88
aniketmaurya
closed
2 weeks ago
3
improve logging
#87
aniketmaurya
closed
2 weeks ago
1
FastAPI background_task cleanup is not executed upon error
#86
aniketmaurya
opened
2 weeks ago
0
format encoded response
#85
aniketmaurya
closed
2 weeks ago
1
automatically dump dictionary to json in encode_response while Streaming
#84
aniketmaurya
closed
2 weeks ago
0
Logging error
#83
williamFalcon
closed
2 weeks ago
2
Let the user know when "setup" has completed...
#82
williamFalcon
closed
2 weeks ago
3
Print "Setup complete" when setup is complete
#81
williamFalcon
closed
2 weeks ago
0
feat: Support multipart & url-encoded form bodies
#80
andyland
closed
3 weeks ago
1
decouple decode_request from get_batch_from_uid
#79
aniketmaurya
closed
3 weeks ago
2
[pre-commit.ci] pre-commit suggestions
#78
pre-commit-ci[bot]
closed
2 weeks ago
1
fix cuda forking error
#77
aniketmaurya
closed
3 weeks ago
1
add openai spec v0
#76
aniketmaurya
closed
1 week ago
9
Bump mypy from 1.9.0 to 1.10.0
#75
dependabot[bot]
closed
1 month ago
1
add license info
#74
aniketmaurya
closed
1 month ago
1
Next