victordibia / llmx

An API for Chat Fine-Tuned Large Language Models (llm)
MIT License
72 stars 24 forks source link

Is the openAI() function call up-to-date with latest args when being initialized? #24

Open alexaios opened 2 months ago

alexaios commented 2 months ago

Hi There,

I found openAI() takes base_url as the mandatory argument to initialize which is mentioned in this vLLM documentation. https://docs.vllm.ai/en/latest/getting_started/quickstart.html#using-openai-completions-api-with-vllm

from openai import OpenAI

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"
client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

However, this base_url is not mentioned in codebase when initializing openAI() https://github.com/victordibia/llmx/blob/main/llmx/generators/text/openai_textgen.py#L30

Should this be updated?

LinkWithMe commented 6 days ago

same problem.