Closed desertaxle closed 2 weeks ago
Should probably be in a separate PR, but I think we'd also benefit from a wait(*all_my_futures)
utility for waiting on arbitrary groups of futures (not necessarily mapped). That being said, I like this interface for mapped futures!
I think we'd also benefit from a
wait(*all_my_futures)
utility for waiting on arbitrary groups of futures
Agreed! I'd like us to have wait
and as_completed
utilities that mirror those in concurrent.futures
in the stdlib. I'll add issues to make sure we remember.
Adds some syntactic sugar to get the results or wait on futures returned from
.map
. The implementation is very naive and will need some iterative improvements, but this makes it feel a big more natural to work with groups of futures.With this PR,
.map
will return aPrefectFutureList
, which walk and quacks likelist
, but also exposes.wait_for_all
and.results
methods. These methods can be used to wait for all futures in thePrefectFutureList
or get their results.Very open to push back on this interface, so please review with a critical eye.
Closes https://github.com/PrefectHQ/prefect/issues/14017
Example
Map a task and wait for all the returned futures:
Map a task and get all the results:
Checklist
maintenance
,fix
,feature
,enhancement
,docs
.<link to issue>
"mint.json
.