stanfordnlp / dspy

DSPy: The framework for programming—not prompting—foundation models
https://dspy-docs.vercel.app/
MIT License
13.82k stars 1.06k forks source link

fix(dspy): fix null reference in settings.trace #1219

Open grant-d opened 2 days ago

grant-d commented 2 days ago

Using dspy-ai===2.4.5 When using the following code:

__llm = dspy.Google(model="models/gemini-1.5-flash")
dspy.settings.configure(lm=__llm) # error below!
dspy.settings.configure(lm=__llm, trace=[]) # mitigates error

# __llm = dspy.OpenAI(model="gpt-4o") # Works fine

class PlannerModule(dspy.Module):
  def forward(self, ...):
    # ... trivial implementation ...
    dspy.Suggest(...)

exec = PlannerModule().activate_assertions(max_backtracks=3)
prediction: dspy.Prediction = exec(input)

I get the following error:

Traceback (most recent call last):
  File "...xyz.py", line 91, in <module>
    prediction: dspy.Prediction = exec(input)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/Data/repos/jigxie/.venv/lib/python3.12/site-packages/dspy/primitives/program.py", line 26, in __call__
    return self.forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/Data/repos/jigxie/.venv/lib/python3.12/site-packages/dspy/primitives/assertions.py", line 318, in forward
    return wrapped_forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/Data/repos/jigxie/.venv/lib/python3.12/site-packages/dspy/primitives/assertions.py", line 241, in wrapper
    dsp.settings.trace.clear()
    ^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'clear'

Not sure why there is a difference in behavior between OpenAI and Google, however, via visual inspection the fix is both defensive & trivial:

okhat commented 17 hours ago

Thank you @grant-d ! I'm not sure this is a correct fix. Why clear the trace altogether?

okhat commented 17 hours ago

i.e., I agree we may need a correction or something but clearing seems not to be the fix?