python-trio / trimeter

(not ready yet) A simple but powerful job scheduler for Trio programs
https://trimeter.readthedocs.io
Other
63 stars 3 forks source link

Multiple iterables? #7

Open njsmith opened 6 years ago

njsmith commented 6 years ago

The builtin map supports multiple iterables:

In [1]: list(map(lambda x, y: x + y, [1, 2, 3], [4, 5, 6]))                     
Out[1]: [5, 7, 9]

(The map in concurrent.futures does too, but multiprocessing.Pool.map does not.)

Should we support multiple iterables?

The obvious argument against it is that it will add some complexity, and it's not clear it's really useful or important.

The other reason to hesitate is that currently we support both sync and async iterables from the same functions, with a cute hack to make this work 99% of the time and an explicit fallback to handle the other 1%... but the explicit fallback is just a single iterable_is_async=True/False argument, so how would we extend that to multple iterables? Does it just apply to all the iterables, so if you pass it you have to make sure your iterables all match? Or do we provide some way to specify on an iterable-by-iterable basis? Or what?

oremanj commented 4 years ago

FWIW, I'd support dropping iterable_is_async entirely, and having the semantics be "async iterable if it has an __aiter__, sync iterable if not". If someone wants an iterable-both-ways to be treated as a sync iterable, they can just wrap it in iter().

If this were a synchronous API, I'd say no need to support multiple args because people can use zip(). But there isn't an async zip() out there for Trio that I know of, and writing a good one is nontrivial (especially if you want to advance both iterables in parallel).

As a bonus, if we support multiple iterables, we can expose a top-level async zip() pretty much for free.

I'd vote for removing iterable_is_async before public release; then we can add multi-iterable support later without deprecating anything if we want to.