Closed chiragshah285 closed 2 months ago
Hi @chiragshah285, to me it seems that it should work (I'm not a maintainer though). Is it not working? Don't know the size of your dataset or what LM you are using, but if you really want to get the best performance, maybe it could make sense to increase the number of samples that LabeledFewShot uses. The default value is 16.
thanks @tom-doerr yeah I was just take 2 examples as a quick test. The problem is more LabelFewShot doesn't support dspy.Suggestion after compilation, from what I see in the code
Hi @chiragshah285 , your code looks correct in terms of using Suggestion within the program. What error do you get while running the code?
DSPy.Suggestions/Assertions are currently compatible with dspy.BootstrapFewShot
and dspy.BootstrapFewShotWithRandomSearch
, so they should work with dspy.LabeledFewShot
as well (which is a candidate within RandomSearch).
thanks @arnavsinghvi11 here is my error message:
Traceback (most recent call last):
File "/Users/chiragshah/PycharmProjects/dspy_test/seo_title_dspy_metrics.py", line 323, in <module>
trained(keyword=KEYWORD,page_content=rp_page_content)
File "/Users/chiragshah/.pyenv/versions/3.11.5/lib/python3.11/site-packages/dspy/primitives/program.py", line 29, in __call__
return self.forward(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/chiragshah/.pyenv/versions/3.11.5/lib/python3.11/site-packages/dspy/primitives/assertions.py", line 317, in forward
return wrapped_forward(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/chiragshah/.pyenv/versions/3.11.5/lib/python3.11/site-packages/dspy/primitives/assertions.py", line 240, in wrapper
dsp.settings.trace.clear()
^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'clear'
ignore this, this works now! thanks for taking a look
Can I use LabeledFewShot with dspy suggestion? I have a bunch of labeled input/output pairs already. I want to compile use that program and do assertions on newly generated outputs. Am I thinking about this the right way? Since I have labeled example already I just need ensure the new outputs from the model are generated per a rule set.
Here is what I'm looking to do: