AzureOpenAI content filtering gives unclear error

[erst-neaste]

Issue

Using dspy-ai==2.4.5, when the AzureOpenAI API's content filtering is triggered, DSPy returns a very unhelpful error.

AttributeError: 'NoneType' object has no attribute 'strip'

From within/site-packages/dsp/primitives/predict.py:78.

Expected behaviour

The AzureOpenAI LM class should throw a clear error stating that the content has been filtered.

[erst-neaste]

Are you open to pull requests here? Happy to make this change myself.

[arnavsinghvi11]

Thanks @erst-neaste, please do contribute PRs!

Could you also share snippets of related code and the full error stack trace? This seems to be an error in configuring your LM with your DSPy program.

[erst-neaste]

This code will replicate the issue:

gpt3 = dspy.AzureOpenAI(
    deployment_id=gpt3_model_name,
    api_key=gpt3_openai_api_key,
    api_base=gpt3_azure_endpoint,
    model_type="chat",
    api_version=openai_api_version,
    temperature=0.1,
    max_tokens=4096,
)

dspy.settings.configure(lm=gpt3)
signature = dspy.Predict('input -> output')
signature(input='Translate "period is over" to danish')

[erst-neaste]

Stack trace:

File "/usr/local/lib/python3.11/site-packages/dspy/predict/predict.py", line 49, in __call__
    return self.forward(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dspy/predict/predict.py", line 91, in forward
    x, C = dsp.generate(template, **config)(x, stage=self.stage)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dsp/primitives/predict.py", line 78, in do_generate
    completions: list[Example] = [template.extract(example, p) for p in completions]
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dsp/primitives/predict.py", line 78, in <listcomp>
    completions: list[Example] = [template.extract(example, p) for p in completions]
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dsp/templates/template_v2.py", line 152, in extract
    raw_pred = raw_pred.strip()
               ^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'strip'

[okhat]

Thanks for opening this! We released DSPy 2.5 yesterday. I think the new dspy.LM and the underlying dspy.ChatAdapter will probably resolve this problem.

Here's the (very short) migration guide, it should typically take you 2-3 minutes to change the LM definition and you should be good to go: https://github.com/stanfordnlp/dspy/blob/main/examples/migration.ipynb

Please let us know if this resolves your issue. I will close for now but please feel free to re-open if the problem persists.