feat(venv): support pip options

bentoml / OpenLLM

Run any open-source LLMs, such as Llama, Mistral, as OpenAI compatible API endpoint in the cloud.

https://bentoml.com

Apache License 2.0

10.12k stars 641 forks source link

Closed rickzx closed 3 months ago

rickzx commented 3 months ago

Support pip options when creating venv. Now requirements.txt can have options like
```
--extra-index-url https://flashinfer.ai/whl/cu121/torch2.3
...
```
Support setting env vars if specified in bento.yaml. This is needed for, e.g., running gemma-2 model with
```
VLLM_ATTENTION_BACKEND=FLASHINFER
```
Reformat all Python files with black

rickzx commented 3 months ago

shouldn't ruff format be sufficient enough?

Yeah, i realized the CI is already doing this. No need for black format