Array like helper functions (combinator module)

[elderapo]

I think it would be handy to have array-like helper functions in the core or utils/helpers package.

const numbersChannel = new Channel<number>(() => {});

const oddNumbersChannel = filterChannel(numbersChannel, item => item % 2 !== 0);
const evenNumbersChannel = filterChannel(numbersChannel, item => item % 2 === 0);

const stringOddNumbersChannel = mapChannel(oddNumbersChannel, item => item.toString());

Can't think of about any other from the top of my head. These should probably be functions and not instance level methods so the API doesn't differ from async iterators.

[brainkim]

I am almost down for this. However:

1. There are multiple libraries that do array-like iterator/async iterator functions.
   • https://github.com/tc39/proposal-iterator-helpers
   • https://github.com/jamiemccrindle/axax
   • https://github.com/fitzgen/wu.js/

Could we use these libraries instead?

2. As a general rule, I want to avoid exporting functions under the @channel namespace which do not use the channel class. Functions like filterChannel could be defined purely with an async generator:

async function *filterAsyncIter<T>(iter: AsyncIterable<T>, pred: (value: T) => any): AsyncIterable<T> {
  for await (const value of iter) {
    if (pred(value)) {
      yield value;
    }
  }
}

Is there a reason we would use channels for these methods?

These should probably be functions and not instance level methods so the API doesn't differ from async iterators.

Yes, my thoughts exactly! 🥰

[elderapo]

I believe these should return channels or utilize some API that allows chaining.

const numbersChannel = new Channel<number>(() => {});

const oddNumbersChannel = filterChannel(numbersChannel, item => item % 2 !== 0);
const oddNumbersGreatherThan50Channel = filterChannel(numbersChannel, item => item > 50);

const newChannel = something(numbersChannel) <-- not idea about the name
    .use(filterChannel, item => item % 2)
    .use(filterChannel, item => item > 50)
    .toChannel();

or

const numbersChannel = new Channel<number>(() => {});

const newChannel = something(numbersChannel)
    .filter(item => item % 2)
    .map(item => `number:${item}`)
    .toChannel();

Also, I think while at util functions we should add ones for converting async generators to channels and vice versa.

[brainkim]

Also, I think while at util functions we should add ones for converting async generators to channels and vice versa.

If we adhere to the “no differences between async generators and channels” rule, we should never have to convert async generators to channels.

Even if you want to create a chainable API, I don’t think there’s a need to have a toChannel call at the end. All you really have to do is make the something function return an async iterable somehow and then use this class interchangeably with channels. This could be a fun project idea!

[elderapo]

Actually, now that I think about this “no differences between async generators and channels” rule it does not make much sense in a sense of extending possibilities of async generators.

The whole point of this package is to make work with "async iterable like" easier no? Channels are essentially an implementation of async iterators. As long as they extend the API of async iterators all the libraries utilizing async iterators should work just fine with channels.

There is zero need for converting channels to async iterators because channels are essentially async iterators. However async iterators are not channels so that's why we should create some asyncGeneratorToChannel utility function.

[brainkim]

Actually, now that I think about this “no differences between async generators and channels” rule it does not make much sense in a sense of extending possibilities of async generators.

Yeah this library isn’t about extending async generators, just making it easy to convert callback-based APIs to async iterator-based APIs. Once you have channels, they are completely indistinguishable from async generator objects.

As long as they extend the API of async iterators all the libraries utilizing async iterators should work just fine with channels.

Yes, exactly.

However async iterators are not channels so that's why we should create some asyncGeneratorToChannel utility function.

This is easy to write with the Channel API:

function convertIterToChan(iter) {
  return new Channel(async (push, stop) => {
    for await (const value of Channel.race([iter, stop])) {
      await push(value);
    }

    stop();
    return stop;
  });
}

But again, my question is, if you already have an async iterator/able, what is the point of upgrading it to a channel? What additional value is derived if you assume that channels are indistinguishable from async generators? Insofar as channels are indistinguishable from async generator objects, you could write an equivalent convertIterToChan with an async generator too!

async function *convertIterToGen(iter) {
  for await (const value of iter) {
    yield value;
  }
}

Or even shorter:

async function *convertIterToGen(iter) {
  yield *iter;
}

The only difference between convertIterToGen and convertIterToChan is that the constructor of the returned iterator is a native generator object in the first case, and this library’s channel class in the second.

[elderapo]

Async generators do not have utilities for manipulating them ex.: filtering, mapping, combining/merging, etc. I believe filtering/mapping can be easily done but more complicated scenarios of merging are much easier with channels.

[brainkim]

Async generators do not have utilities for manipulating them ex.: filtering, mapping, combining/merging, etc. I believe filtering/mapping can be easily done but more complicated scenarios of merging are much easier with channels.

Channels will never have utilities for filtering/mapping unless async generators gain these methods as well. Additionally, the channel combinator functions make a conscious decision to only take AsyncIterable values as parameters; they never require their arguments to be channels. Typically, everything you need for filtering, mapping, etc. is already exposed via the AsyncIterable interface. In your above example, I’m not sure why or how intermediate channels would be used to implement map/filter, and I guarantee those things can be done with async generators (and you can create a fluent chainable API with a little bit of extra work).

I think there is definitely a need for a library which does what you’re saying, and it would be even better if it were both typed with TypeScript and polymorphically sync/async according to whether the arguments passed to it are Iterable or AsyncIterable. I encourage you to experiment with this stuff and let me know what you discover!

[brainkim]

Reopening this cuz I plan on moving the combinator static method to a separate module and adding combinators.