[Question]: VectorStoreIndex.as_query_engine() with OpenAI compatible API

JulianOestreich90 commented 5 months ago

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question

I am using an LLM from an OpenAI compatible API and I load it as a Llangchain LLM:

import os
from langchain.llms import OpenAI
from llama_index.llms.langchain import LangChainLLM

os.environ["OPENAI_API_KEY"] = "hjkbgjk"
os.environ["OPENAI_API_BASE"] = "https://helmholtz-blablador.fz-juelich.de:8000/v1"

llm = LangChainLLM(llm=OpenAI(model="alias-fast", temperature=0.0))

The model works perfectly fine with

 llm.complete("Hey how are you?")

However if i am trying to initialize a query engine from a VectorStoreIndex like this:

 query_engine_vector = vector_index.as_query_engine(
    llm=llm,
    similarity_top_k=5,
    filters=MetadataFilters(
        filters=[
            ExactMatchFilter(key="file_name", value="doc_to_filter"),
        ]
    ),
)

I am getting the error:

ValueError: Unknown model 'alias-fast'. Please provide a valid OpenAI model name in: gpt-4, gpt-4-32k, gpt-4-1106-preview, gpt-4-0125-preview, gpt-4-turbo-preview, gpt-4-vision-preview, gpt-4-turbo-2024-04-09, gpt-4-turbo, gpt-4-0613, gpt-4-32k-0613, gpt-4-0314, gpt-4-32k-0314, gpt-3.5-turbo, gpt-3.5-turbo-16k, gpt-3.5-turbo-0125, gpt-3.5-turbo-1106, gpt-3.5-turbo-0613, gpt-3.5-turbo-16k-0613, gpt-3.5-turbo-0301, text-davinci-003, text-davinci-002, gpt-3.5-turbo-instruct, text-ada-001, text-babbage-001, text-curie-001, ada, babbage, curie, davinci, gpt-35-turbo-16k, gpt-35-turbo, gpt-35-turbo-0125, gpt-35-turbo-1106, gpt-35-turbo-0613, gpt-35-turbo-16k-0613

dosubot[bot] commented 5 months ago

The error you're encountering is due to the model name 'alias-fast' not being recognized. This name does not match any of the models listed in the system's ALL_AVAILABLE_MODELS dictionary, which includes models from GPT-4, GPT-3.5 Turbo, and other supported models.

To resolve this issue, ensure you use a model name that is recognized by the system, such as "gpt-4", "gpt-3.5-turbo", "text-davinci-003", or any other valid model name from the supported list. If 'alias-fast' is an alias or a custom model, you'll need to map it to a supported model name or adjust your implementation to recognize it correctly.

For more details on supported models and how to properly configure them, you might want to check the documentation or the relevant code sections in the repository, specifically around model name handling and the openai_modelname_to_contextsize function.

Sources

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

JulianOestreich90 commented 5 months ago

But i want to access other models through an OpenAI compatible API. The API i want to use is Helmholtz Blablador.

logan-markewich commented 5 months ago

@JulianOestreich90 I would use the OpenAILike class for this

pip install llama-index-llms-openai-like

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
  model="alias-fast", 
  api_base=https://helmholtz-blablador.fz-juelich.de:8000/v1",
  api_key="fake",
  temperature=0.0,
  # I don't actually know if this is true or false for you -- is the messages api supported for your model?
  is_chat_model=True, 
)

run-llama / llama_index