Lightning-AI / LitServe

Lightning-fast serving engine for any AI model of any size. Flexible. Easy. Enterprise-scale.
https://lightning.ai/docs/litserve
Apache License 2.0
2.5k stars 158 forks source link

install uvloop with uvcorn #274

Closed aniketmaurya closed 2 months ago

aniketmaurya commented 2 months ago
Before submitting - [ ] Was this discussed/agreed via a Github issue? (no need for typos and docs improvements) - [ ] Did you read the [contributor guideline](https://github.com/Lightning-AI/pytorch-lightning/blob/main/.github/CONTRIBUTING.md), Pull Request section? - [ ] Did you make sure to update the docs? - [ ] Did you write any new necessary tests?

⚠️ How does this PR impact the user? ⚠️

Avoid uvloop not installed warning by automatic installation on supported system.

GOOD:

"As a user, I need to serve models faster. This PR focuses on enabling speed gains by using GPUs"

BAD:

"This PR enables GPUs". This is bad because the user problem is not clear... instead it just jumps to the solution without any context.

PRs without this will not be merged.


What does this PR do?

uvloop is a faster drop-in replacement of asyncio loop and we used it during all our benchmarks. It should be shipped along with the package on supported systems.

uvcorn[standard] installs uvloop.

PR review

Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

codecov[bot] commented 2 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95%. Comparing base (1e7140d) to head (5e62946). Report is 1 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #274 +/- ## =================================== Coverage 95% 95% =================================== Files 14 14 Lines 1082 1082 =================================== + Hits 1023 1025 +2 + Misses 59 57 -2 ```