(Feature) Async Support

[CyrusOfEden]

🥁 the long awaited async implementation for DSPy 🔥

All dspy.predict.* programs can now be used async!

dspy.settings.configure(async_mode=True)

All programs are now awaitable:

predict = dspy.Predict("question -> answer")
result = await predict(question="...")
assert result.answer == "..."

Custom programs must implement the aforward method, which has the same method signature as forward. Developers must take care not to block the event loop with synchronous function calls.

class MyProgram(dspy.Module):
  async def aforward(...):
    ...

Notable Changes

0. Added a @with_async_callbacks impl based off of @dbczumar's @with_callbacks impl
1. Retrievers now have a default async implementation that delegates to a threadpool
2. New LRU cache for completions, with a default maxlen of 10M. Can be overwritten with: dspy.settings.configure(request_cache=LMRequestLRUCache(maxsize: int = ...). This prevents unbounded memory growth.
3. GLOBAL_HISTORY updated from a list to a deque with a maxlen of 10M, to prevent unbounded memory growth.

Minor Changes

1. Request caching previously serialized, deserialized, then serialize the request JSON. I've removed the unnecessary deserialization step.

Notes

It looks like the ReAct tests are broken in main, and they're broken here too.

Acknowledgements

Kudos to @isaacbmiller for his prototype in #1729, this wouldn't have been possible without him!

[okhat]

This is AWESOME, thanks a lot @CyrusOfEden !!

Are you using a linter that's more opinionated than the default linting behavior in the library? I think a large fraction of the 24 edited files are just edited for style and that makes review / history a bit more complex than it needs to be for this PR.

Relatedly, there are some outdated ReAct tests that were commented out, we can keep them commented out until they're updated.

[okhat]

tl;dr

1. I'd love to review and merge a version of this PR that focuses on content updates. If you prefer, we could merge a (less aggressive) style reformat PR first for the relevant files.
2. For us to merge, we need to consider the implications on complex programs that involve, e.g. with dspy.context calls. Alternatively, I'm fine disallowing any .context and .configure calls in async programs as long as it throws an error, so people don't forget.

[CyrusOfEden]

I think a large fraction of the 24 edited files are just edited for style

Only files that have had changes have had the formatting applied, so there's no superfluous formatting included in the PR.

My editor should've picked up the formatting rules, I can check in the afternoon today.

Edit: See latest push

[ArslanS1997]

Hi how can I use this, is this merged?

[CyrusOfEden]

Closing this in favour of #1806