vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
26.78k stars 3.92k forks source link

[Feature]: Add security scheme to server #6970

Open g-parki opened 1 month ago

g-parki commented 1 month ago

🚀 The feature, motivation and pitch

Hi there! I'd be happy to submit a PR for this, but wanted to open an issue for discussion first. The current method for authenticating with the server API key is undocumented in the auto-generated openapi.json. This breaks the Swagger /docs page because there is no functionality to login or otherwise provide the authorization header.

As a solution, FastAPI has some out-of-the-box functionality for adding a security scheme (namely fastapi.security.HTTPBearer) which, once included in an app/router/route, is automatically applied to the OpenAPI spec. This fixes the Swagger docs as well. My suggested approach is:

Here's a minimal proof-of-concept. Click to expand. ``` python import os from typing import Annotated from dotenv import load_dotenv from fastapi import FastAPI, Depends, HTTPException, status, APIRouter from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials load_dotenv() # GET API KEY FROM ENVIRONMENT OR ARGS API_KEY = os.getenv("API_KEY") token_auth_scheme = HTTPBearer(scheme_name="ApiKeyAuth") # FUNCTION TO VALIDATE A CREDENTIAL, USES BUILT-IN FASTAPI FUNCTIONALITY def verify_token(auth_credentials: Annotated[HTTPAuthorizationCredentials, Depends(token_auth_scheme)]) -> None: if API_KEY is not None and auth_credentials.credentials != API_KEY: raise HTTPException( status_code=status.HTTP_403_FORBIDDEN, detail={"error": "Unauthorized"} ) # CREATE TWO ROUTERS - ONE WITH NO AUTHENTICATION, AND ONE WHICH MAY HAVE AUTHENTICATION app = FastAPI() router = APIRouter() auth_required_router = APIRouter( dependencies=[Depends(verify_token)] if API_KEY else [] ) # USE EACH ROUTER @router.get('/health') def health(): return {"message": "This is an unprotected route, even if the API key is set"} @auth_required_router.get("/protected-endpoint") def protected_endpoint(): return {"message": "This route is protected if the API key is set"} @auth_required_router.get("/another") def another_endpoint(): return {"message": "This is also protected if the API key is set"} app.include_router(router) app.include_router(auth_required_router) ```

When the server is started with an API key, the Swagger docs include an Authorize button and shows which routes are secured:

image

Alternatives

The app can also have additional OpenAPI info added to it manually, but I don't view this is as a good practice when there are built-in classes to handle this task.

Additional context

I verified the proof-of-concept code produces a security scheme which matches the OpenAI OpenAPI security schema

simon-mo commented 1 month ago

I think this is a good idea. Thank you bringing it up and looking forward towards your PR!