stanfordnlp / dspy

DSPy: The framework for programming—not prompting—language models
https://dspy.ai
MIT License
19.27k stars 1.47k forks source link

Add support for native parallel execution in DSPy #1690

Closed krypticmouse closed 1 week ago

krypticmouse commented 1 month ago

This pull request introduces significant refactoring and new functionality to the dspy package. The most important changes include the introduction of a ParallelExecutor class, updates to the Evaluate class to use this new executor, and the addition of a batch method in the Module class.

Parallel Execution Enhancements:

New Functionality:

okhat commented 1 month ago

One important note: I think we should deepcopy the stack when we're forking a thread! Not just copy, but deep copy, because the lists in the values are mutable.

okhat commented 1 month ago

Also important: we need to handle creating threads inside threads. This means every thread needs to "inherit" / fork / deepcopy the stack from its parent, not from the main stack.

This was not as crucial before because there was no case where people nested parallelism! Now with dspy.Parallel() everyone will nest parallelism.

ArslanS1997 commented 1 week ago

Hi do I use dspy.Parallel?