[Bug]: GoogleDemo.ipynb notebook seems to be requiring OpenAI API key

mariohinojosa commented 4 months ago

Bug Description

Hi,

I'm following the demo: https://github.com/run-llama/llama_index/blob/2f2d5a4735dd82f8acd2f630c745758e92202be2/docs/docs/examples/managed/GoogleDemo.ipynb

I have configured service account auth, enabled the Google API, downloaded the paul graham doc, but when I get to the basic querying section I get the following error:

ValueError:

Could not load OpenAI model. If you intended to use OpenAI, please check your OPENAI_API_KEY. Original error: No API key found for OpenAI. Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization. API keys can be found or created at https://platform.openai.com/account/api-keys

To disable the LLM entirely, set llm=None.

I was under the impression that the LLM in this case would be Google's Gemini and the necessary credentials were already set by passing the Service Account details to set_google_config(auth_credentials=credentials)

Version

llama_index-0.10.28

Steps to Reproduce

Follow the instructions on the notebook

Relevant Logs/Tracbacks

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/llama_index/core/llms/utils.py in resolve_llm(llm, callback_manager)
     40             llm = OpenAI()
---> 41             validate_openai_api_key(llm.api_key)
     42         except ImportError:

6 frames
ValueError: No API key found for OpenAI.
Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization.
API keys can be found or created at https://platform.openai.com/account/api-keys

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/llama_index/core/llms/utils.py in resolve_llm(llm, callback_manager)
     46             )
     47         except ValueError as e:
---> 48             raise ValueError(
     49                 "\n******\n"
     50                 "Could not load OpenAI model. "

ValueError: 
******
Could not load OpenAI model. If you intended to use OpenAI, please check your OPENAI_API_KEY.
Original error:
No API key found for OpenAI.
Please set either the OPENAI_API_KEY environment variable or openai.api_key prior to initialization.
API keys can be found or created at https://platform.openai.com/account/api-keys

To disable the LLM entirely, set llm=None.
******

dosubot[bot] commented 4 months ago

It appears the issue arises because LlamaIndex is attempting to initialize an OpenAI model by default, which requires an OpenAI API key. Since you're aiming to use Google's Gemini LLM, you need to explicitly specify this to avoid the default behavior.

To resolve this, ensure you're specifying Google's Gemini LLM correctly in your setup. If the documentation or the example notebook you're following doesn't explicitly mention how to set Google's LLM as the default, you might need to look into the LLMPredictor initialization or wherever the LLM is being configured in your code.

If the intention is to bypass the use of an OpenAI model entirely, you could set llm=None to use a mock model, which would prevent the code from attempting to load an OpenAI model and thus avoid the error related to the missing API key.

In case these steps don't align with your setup or if the error persists, consider setting a dummy OPENAI_API_KEY in your environment to bypass the check as a temporary workaround:

import os
os.environ["OPENAI_API_KEY"] = 'dummy'

This is a workaround and may not be suitable for all scenarios, so it's recommended to review the codebase, especially around the model initialization, to ensure compatibility with your intended setup.

For further assistance, you might find it helpful to refer to discussions or issues within the LlamaIndex repository that relate to customizing the LLM or resolving API key errors.

Sources

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

logan-markewich commented 4 months ago

It still needs an LLM to do response synthesis. It doesn't do that for you, and instead defaults to the global settings

You should define an LLM, and either change the global defaults

from llama_index.core import Settings
Settings.llm = llm

Or pass it in (I'm assuming its this line of code, although you didn't specify) query_engine = index.as_query_engine(llm=llm)

mariohinojosa commented 4 months ago

Thanks a lot for the prompt reply, I really appreciate it. I thought the demo's instructions were self-contained, meaning that there were no "inferred" actions or pre-requisites for the user, but now I know this is not the case.

I have now tried your suggestion using both OpenAI and Gemini LLMs. In both cases I get a very cryptic AssertionError:

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
[<ipython-input-39-d3d2ef269f16>](https://localhost:8080/#) in <cell line: 17>()
     15 
     16 query_engine = index.as_query_engine()
---> 17 response = query_engine.query("What did Paul Graham do growing up?")
     18 assert isinstance(response, Response)
     19 

6 frames
[/usr/local/lib/python3.10/dist-packages/llama_index/vector_stores/google/genai_extension.py](https://localhost:8080/#) in generate_answer(prompt, passages, answer_style, safety_settings, temperature, client)
    549         )
    550 
--> 551     assert len(response.answer.content.parts) == 1
    552     return GroundedAnswer(
    553         answer=response.answer.content.parts[0].text,

AssertionError:

The code block I am executing is the following:

#============== START IMPLEMENTED SUGGESTION ==============
# Import libraries
from llama_index.core import Settings
from llama_index.llms.openai import OpenAI
from llama_index.llms.gemini import Gemini
from google.colab import userdata
import os

# load variables into environment
os.environ["OPENAI_API_KEY"] = userdata.get('OPENAI_API_KEY')
# os.environ["GOOGLE_API_KEY"] = userdata.get('GOOGLE_API_KEY')

# Define LLM global setting
Settings.llm = OpenAI( model=f"gpt-3.5-turbo")
# Settings.llm = Gemini(model_name="models/gemini-1.0-pro")

#============== END IMPLEMENTED SUGGESTION ==============

# Querying
query_engine = index.as_query_engine()
response = query_engine.query("What did Paul Graham do growing up?")
assert isinstance(response, Response)

# Show response.
print(f"Response is {response.response}")

# Show cited passages that were used to construct the response.
for cited_text in [node.text for node in response.source_nodes]:
    print(f"Cited text: {cited_text}")

# Show answerability. 0 means not answerable from the passages.
# 1 means the model is certain the answer can be provided from the passages.
if response.metadata:
    print(
        f"Answerability: {response.metadata.get('answerable_probability', 0)}"
    )

I don't know if what I'm about to share is related to this issue, but I have noticed that in the notebook's previous block, where the index is created and populated, it returns a Warning message. The code and message are the following:

# PREVIOUS CODE BLOCK
from llama_index.core import SimpleDirectoryReader
from llama_index.indices.managed.google import GoogleIndex
from llama_index.core import Response
import time

# Create a corpus.
index = GoogleIndex.create_corpus(
    corpus_id=SESSION_CORPUS_ID, display_name="My first corpus!"
)
print(f"Newly created corpus ID is {index.corpus_id}.")

# Ingestion.
documents = SimpleDirectoryReader("./data/paul_graham/").load_data()
index.insert_documents(documents)

# RETURNED ERROR
Newly created corpus ID is llama-index-colab-77875-1.
WARNING:llama_index.vector_stores.google.genai_extension:Document 851d8f88-cc7f-4d87-afa2-ea5a5322e0e1 in corpus llama-index-colab-77875-1 not found: 404 Documents does not exist```

Any ideas what this AssertionError is about?

logan-markewich commented 4 months ago

Seems like an issue with the response coming from googles API 😞

run-llama / llama_index