Open thomasahle opened 1 month ago
I closed due to some breaking changes, but if it's interesting, would you be able and interested in reviving the PR.
I'm not familiar with the instructor codebase. But I'll have a look!
Ok, I think you are right that Anthropic has to be excluded, as they don't support it. I guess an option would be to just do multiple calls in this case, but maybe it's better to throw an error.
Otherwise, I don't see any issues with your PR!
Is your feature request related to a problem? Please describe. I'm trying to do something like Getting 50% (SoTA) on ARC-AGI with GPT-4o, and it requires drawing thousands of independent samples.
Describe the solution you'd like I'm basically trying to reopen https://github.com/jxnl/instructor/issues/576 since it seems it was closed together with the pull request https://github.com/jxnl/instructor/pull/578 because it wasn't clear "if people really needed this".
This means instructor should support the
n=...
argument as in:which would return a
list[Foo]
with 3 objects.Describe alternatives you've considered Say my output type is:
I could instead use
to force the model to give me 5 outputs.
However, this might overflow the output context; and it would make the outputs "dependent" when what I want are independent samples.
I could also do
n
independent calls toclient.chat.completions.create
, but when I have a very large prompt, this is very expensive.