search

huggingface / text-generation-inference

Large Language Model Text Generation Inference
http://hf.co/docs/text-generation-inference
Apache License 2.0
8.36k stars 948 forks source link

API_KEY argument #2026

Open nbroad1881 opened 1 month ago

nbroad1881 commented 1 month ago

Feature request

TEI has the API_KEY argument:

   --api-key <API_KEY>
      Set an api key for request authorization.

      By default the server responds to every request. With an api key set, the requests must have the Authorization header set with the api key as Bearer token.

      [env: API_KEY=]

It would be nice if TGI had this as well.

Motivation

It enables custom authorization of the endpoint.

Your contribution

sry

KevinDuffy94 commented 1 week ago

Adding this block in here and appending the layer should achieve this.

One negative effect of this is that the healthcheck and info endpoints get auth included.

I would propose having a slightly different approach to TEI and only adding the Auth Layer at the route level for non information endpoints. I could start a PR for this next week.

KevinDuffy94 commented 1 week ago

https://github.com/huggingface/text-generation-inference/pull/2142