search

stanford-oval / storm

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.
http://storm.genie.stanford.edu
MIT License
10.12k stars 960 forks source link

Unable to use Azure OpenAI #86

Closed Justincjr closed 1 week ago

Justincjr commented 1 month ago

I have provided the api key, endpoint and the api version and encountered an error. Im pretty sure azure openai requires the deployment name of the model, are there anyways to fix this bug? Thank you

Yucheng-Jiang commented 1 month ago

@Justincjr Yes you are right, when using azure openai, you need to make sure deployment name matches with the default name we put in the code base. Or you can adjust the name in the code as you needed.

If you would like us to troubleshoot more, more context and error messages would be greatly appreciated.

Justincjr commented 1 month ago

openai.AuthenticationError: Error code: 401 - {'error': {'message': 'Incorrect API key provided: d579c5a5****3882. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}

I already selected azure as api type, and changed the model name in the source code

naseerfaheem commented 1 month ago

@Justincjr I face the exact issue with Azure OpenAI. I picked "azure" in the API type, provided end point and api verison, but keep getting the invalid api key error.

Yucheng-Jiang commented 1 month ago

Are you still facing this problem? If so would you mind provide detailed steps to reproduce this issue so that we can assist further.

Yucheng-Jiang commented 1 month ago

Will close this issue if no further question on this thread.

Justincjr commented 1 month ago

Solved this problem, but facing a bunch of errors with bing search

ericthomas1 commented 4 weeks ago

I'm unable to use AzureOpenAI + Bing Search as well. Spent a lot of time trying to customize front-end/demo_light to no avail.

Latest error message:

(.venv) PS C:\Users\redacto\Documents\projects\storm\frontend\demo_light> streamlit run .\storm.py

  You can now view your Streamlit app in your browser.

  Local URL: http://localhost:8501
  Network URL: http://10.11.102.214:8501

root : WARNING  : No valid OpenAI API provider is provided. Cannot use default LLM configurations.
root : WARNING  : Language model for conv_simulator_lm is not initialized. Please call set_conv_simulator_lm()
root : WARNING  : Language model for outline_gen_lm is not initialized. Please call set_outline_gen_lm()
root : WARNING  : Language model for article_gen_lm is not initialized. Please call set_article_gen_lm()
root : WARNING  : Language model for article_polish_lm is not initialized. Please call set_article_polish_lm()
2024-08-15 16:20:01.614 Uncaught app exception
Traceback (most recent call last):
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 535, in _run_script
    exec(code, module.__dict__)
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\storm.py", line 60, in <module>
    main()
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\storm.py", line 56, in main
    CreateNewArticle.create_new_article_page()
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\pages_util\CreateNewArticle.py", line 59, in create_new_article_page
    st.session_state["runner"].run(
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\knowledge_storm\storm_wiki\engine.py", line 293, in run
    information_table = self.run_knowledge_curation_module(ground_truth_url=ground_truth_url,
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\knowledge_storm\interface.py", line 376, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\knowledge_storm\storm_wiki\engine.py", line 165, in run_knowledge_curation_module
    information_table, conversation_log = self.storm_knowledge_curation_module.research(
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\knowledge_storm\storm_wiki\modules\knowledge_curation.py", line 306, in research
    considered_personas = self._get_considered_personas(topic=topic, max_num_persona=max_perspective)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\knowledge_storm\storm_wiki\modules\knowledge_curation.py", line 231, in _get_considered_personas
    return self.persona_generator.generate_persona(topic=topic, max_num_persona=max_num_persona)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\knowledge_storm\storm_wiki\modules\persona_generator.py", line 135, in generate_persona
    personas = self.create_writer_with_persona(topic=topic)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\dspy\primitives\program.py", line 26, in __call__
    return self.forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\knowledge_storm\storm_wiki\modules\persona_generator.py", line 70, in forward
    related_topics = self.find_related_topic(topic=topic).related_topics
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\dspy\predict\predict.py", line 61, in __call__
    return self.forward(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\dspy\predict\chain_of_thought.py", line 59, in forward
    return super().forward(signature=signature, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\dspy\predict\predict.py", line 103, in forward
    x, C = dsp.generate(template, **config)(x, stage=self.stage)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\dsp\primitives\predict.py", line 77, in do_generate
    completions: list[dict[str, Any]] = generator(prompt, **kwargs)
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\dsp\modules\azure_openai.py", line 199, in __call__
    response = self.request(prompt, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\backoff\_sync.py", line 105, in retry     
    ret = target(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\dsp\modules\azure_openai.py", line 171, in request
    return self.basic_request(prompt, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\dsp\modules\azure_openai.py", line 144, in basic_request
    response = chat_request(self.client, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\dsp\modules\azure_openai.py", line 302, in chat_request
    return v1_chat_request(client, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\dsp\modules\azure_openai.py", line 282, in v1_chat_request
    return v1_cached_gpt3_turbo_request_v2_wrapped(**kwargs).model_dump()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\dsp\modules\cache_utils.py", line 16, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\dsp\modules\azure_openai.py", line 280, in v1_cached_gpt3_turbo_request_v2_wrapped
    return v1_cached_gpt3_turbo_request_v2(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\joblib\memory.py", line 655, in __call__  
    return self._cached_call(args, kwargs)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\joblib\memory.py", line 598, in _cached_call
    out, metadata = self.call(*args, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\joblib\memory.py", line 856, in call      
    output = self.func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\dsp\modules\azure_openai.py", line 278, in v1_cached_gpt3_turbo_request_v2
    return client.chat.completions.create(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\redacto\Documents\projects\storm\frontend\demo_light\.venv\Lib\site-packages\openai\_utils\_utils.py", line 274, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
TypeError: Completions.create() got an unexpected keyword argument 'api_provider'

FIXED: In this case, I didn't correctly customize front-end/demo_light/demo_utils.py. Commented-out the api_provider as shown here. This fixed the error above, but still unable to use as shown in subsequent posts.

image

Yucheng-Jiang commented 4 weeks ago

could you verify you have correctly configured front-end/demo_light/.stremlit/secrets.toml as describe as third bullet point under the Setup section in Readme?

ericthomas1 commented 3 weeks ago

Yes, I have the secrets.toml placed in the correct directory as shown below.

What is the correct URL pattern for Azure OpenAI (the AZURE_API_BASE environment variable)? 

# Set up *Azure* OpenAI
OPENAI_API_KEY="<redacted>"
OPENAI_API_TYPE="azure"

** Is this the correct URL pattern? **
AZURE_API_BASE="https://<my-instance>.openai.azure.com/openai/deployments/gpt-4o/"

AZURE_API_VERSION="2024-05-13"
BING_SEARCH_API_KEY="<redacted>"
ericthomas1 commented 3 weeks ago

After trying various patterns for the AZURE_API_BASE value, I'm now receiving these errors when Creating a New Article:

Environment variable: AZURE_API_BASE="<my-instance>"

root : WARNING  : No valid OpenAI API provider is provided. Cannot use default LLM configurations.
root : WARNING  : Language model for conv_simulator_lm is not initialized. Please call set_conv_simulator_lm()
root : WARNING  : Language model for outline_gen_lm is not initialized. Please call set_outline_gen_lm()
root : WARNING  : Language model for article_gen_lm is not initialized. Please call set_article_gen_lm()
root : WARNING  : Language model for article_polish_lm is not initialized. Please call set_article_polish_lm()
openai._base_client : INFO     : Retrying request to /chat/completions in 0.970605 seconds
openai._base_client : INFO     : Retrying request to /chat/completions in 1.520660 seconds
backoff : INFO     : Backing off request(...) for 0.1s (openai.APIConnectionError: Connection error.)
Backing off 0.1 seconds after 1 tries calling function <function AzureOpenAI.request at 0x000002A39D6C2A20> with kwargs {}
openai._base_client : INFO     : Retrying request to /chat/completions in 0.883230 seconds
openai._base_client : INFO     : Retrying request to /chat/completions in 1.759028 seconds
backoff : INFO     : Backing off request(...) for 1.2s (openai.APIConnectionError: Connection error.)
Backing off 1.2 seconds after 2 tries calling function <function AzureOpenAI.request at 0x000002A39D6C2A20> with kwargs {}
openai._base_client : INFO     : Retrying request to /chat/completions in 0.986636 seconds
openai._base_client : INFO     : Retrying request to /chat/completions in 1.898186 seconds
backoff : INFO     : Backing off request(...) for 3.6s (openai.APIConnectionError: Connection error.)
Backing off 3.6 seconds after 3 tries calling function <function AzureOpenAI.request at 0x000002A39D6C2A20> with kwargs {}
openai._base_client : INFO     : Retrying request to /chat/completions in 0.762531 seconds
openai._base_client : INFO     : Retrying request to /chat/completions in 1.688774 seconds
backoff : INFO     : Backing off request(...) for 0.9s (openai.APIConnectionError: Connection error.)
Backing off 0.9 seconds after 4 tries calling function <function AzureOpenAI.request at 0x000002A39D6C2A20> with kwargs {}
openai._base_client : INFO     : Retrying request to /chat/completions in 0.855033 seconds
  Stopping...
openai._base_client : INFO     : Retrying request to /chat/completions in 1.838988 seconds
backoff : INFO     : Backing off request(...) for 0.2s (openai.APIConnectionError: Connection error.)
Backing off 0.2 seconds after 5 tries calling function <function AzureOpenAI.request at 0x000002A39D6C2A20> with kwargs {}
openai._base_client : INFO     : Retrying request to /chat/completions in 0.889916 seconds
openai._base_client : INFO     : Retrying request to /chat/completions in 1.523049 seconds
backoff : INFO     : Backing off request(...) for 23.7s (openai.APIConnectionError: Connection error.)
Backing off 23.7 seconds after 6 tries calling function <function AzureOpenAI.request at 0x000002A39D6C2A20> with kwargs {}
Yucheng-Jiang commented 3 weeks ago

@ericthomas1 I assume you're dealing with demo_light? 

root : WARNING  : No valid OpenAI API provider is provided

From this error it seems like secrets.toml is not properly loaded. I would suggest adding some debug message here like assert os.getenv("OPENAI_API_TYPE") to verify this.

ericthomas1 commented 3 weeks ago

@Yucheng-Jiang , thank you for the guidance. I can see the environment variable, but error persists.

Debug logging: 

import os
import logging

script_dir = os.path.dirname(os.path.abspath(__file__))
wiki_root_dir = os.path.dirname(os.path.dirname(script_dir))

import demo_util
from pages_util import MyArticles, CreateNewArticle
from streamlit_float import *
from streamlit_option_menu import option_menu

def main():
    global database
    st.set_page_config(layout='wide')

    if "first_run" not in st.session_state:
        st.session_state['first_run'] = True

    # set api keys from secrets
    if st.session_state['first_run']:
        for key, value in st.secrets.items():
            if type(value) == str:
                os.environ[key] = value

    logging.info(f'###  OPENAI_API_TYPE: {os.getenv("OPENAI_API_TYPE")}')

...

Results:

root : INFO     : ###  OPENAI_API_TYPE: azure
root : INFO     : ###  OPENAI_API_TYPE: azure
root : INFO     : ###  OPENAI_API_TYPE: azure
root : INFO     : ###  OPENAI_API_TYPE: azure
root : INFO     : ###  OPENAI_API_TYPE: azure
root : INFO     : ###  OPENAI_API_TYPE: azure
root : WARNING  : No valid OpenAI API provider is provided. Cannot use default LLM configurations.
root : WARNING  : Language model for conv_simulator_lm is not initialized. Please call set_conv_simulator_lm()
root : WARNING  : Language model for outline_gen_lm is not initialized. Please call set_outline_gen_lm()
root : WARNING  : Language model for article_gen_lm is not initialized. Please call set_article_gen_lm()
root : WARNING  : Language model for article_polish_lm is not initialized. Please call set_article_polish_lm()
openai._base_client : INFO     : Retrying request to /chat/completions in 0.771562 seconds
openai._base_client : INFO     : Retrying request to /chat/completions in 1.966350 seconds
backoff : INFO     : Backing off request(...) for 0.7s (openai.APIConnectionError: Connection error.)
Backing off 0.7 seconds after 1 tries calling function <function AzureOpenAI.request at 0x00000240543F2D40> with kwargs {}
openai._base_client : INFO     : Retrying request to /chat/completions in 0.830957 seconds
openai._base_client : INFO     : Retrying request to /chat/completions in 1.894098 seconds
backoff : INFO     : Backing off request(...) for 1.5s (openai.APIConnectionError: Connection error.)
Backing off 1.5 seconds after 2 tries calling function <function AzureOpenAI.request at 0x00000240543F2D40> with kwargs {}
openai._base_client : INFO     : Retrying request to /chat/completions in 0.993987 seconds
openai._base_client : INFO     : Retrying request to /chat/completions in 1.738715 seconds
ericthomas1 commented 3 weeks ago

@Yucheng-Jiang , Upon investigating this further, storm/knowledge_storm/storm_wiki/engine.py will not work with azure, correct?

Seems the error No valid OpenAI API provider is provided. Cannot use default LLM configurations. originates at this point.

image

Yucheng-Jiang commented 3 weeks ago

ah I see, previously we remove default support for Azure OpenAI due to package upgrade. We will add it back sometime later. For now, following code would be helpful. In this function change to following

    def init_openai_model(
            self,
            openai_api_key: str,
            azure_api_key: str,
            openai_type: Literal["openai", "azure"],
            api_base: Optional[str] = None,
            api_version: Optional[str] = None,
            temperature: Optional[float] = 1.0,
            top_p: Optional[float] = 0.9
    ):
        azure_kwargs = {
            "api_key": azure_api_key,
            "temperature": temperature,
            "top_p": top_p,
            "api_base": api_base,
            "api_version": api_version
        }

        openai_kwargs = {
            "api_key": openai_api_key,
            "api_provider": "openai",
            "temperature": temperature,
            "top_p": top_p,
            "api_base": None,
        }
        if openai_type and openai_type == 'openai':
            self.conv_simulator_lm = OpenAIModel(model='gpt-4o-2024-05-13',
                                                 max_tokens=500, **openai_kwargs)
            self.question_asker_lm = OpenAIModel(model='gpt-4o-2024-05-13',
                                                 max_tokens=500, **openai_kwargs)
            self.outline_gen_lm = OpenAIModel(model='gpt-4o-2024-05-13',
                                              max_tokens=400, **openai_kwargs)
            self.article_gen_lm = OpenAIModel(model='gpt-4o-2024-05-13',
                                              max_tokens=700, **openai_kwargs)
            self.article_polish_lm = OpenAIModel(model='gpt-4o-2024-05-13',
                                                 max_tokens=4000, **openai_kwargs)
        elif openai_type and openai_type == 'azure':
            self.conv_simulator_lm = AzureOpenAIModel(model='gpt-4o-2024-05-13', max_tokens=500,
                                                      **azure_kwargs, model_type="chat")
            self.question_asker_lm = AzureOpenAIModel(model='gpt-4o-2024-05-13', max_tokens=500,
                                                      **azure_kwargs, model_type="chat")
            self.outline_gen_lm = AzureOpenAIModel(model='gpt-4o-2024-05-13', max_tokens=400,
                                                   **azure_kwargs, model_type="chat")
            self.article_gen_lm = AzureOpenAIModel(model='gpt-4o-2024-05-13', max_tokens=700,
                                                   **azure_kwargs, model_type="chat")
            self.article_polish_lm = AzureOpenAIModel(model='gpt-4o-2024-05-13, max_tokens=4000,
                                                      **azure_kwargs, model_type="chat")
        else:
            logging.warning('No valid OpenAI API provider is provided. Cannot use default LLM configurations.')

You can change model type as needed (gpt-4o works reasonable well empirically); and azure openai model name should also match with your own deployment name.

Hope it helps.