Boolean literal types and return type propagation for generators

[JsonFreeman]

This suggestion has a few pieces:

1. Implement boolean literal types for true and false, in a fashion similar to #1003
2. Implement type guards for booleans. Essentially, control flow constructs like if-else, while, for, etc would be subject to a type guard if their guard expression is a boolean.
3. Now we can update generators to have a next method that returns { done: false; value: TYield; } | { done: true; value: TReturn; }, where TYield is inferred from the yield expressions of a generator, and TReturn is inferred from the return expressions of a generator.
4. Iterators would return { done: false; value: TYield; } | { done: true; value: any; } to be compatible with generators.
5. for-of, spread, array destructuring, and the type yielded by yield* would only pick the value associated with done being false.
6. The value of a yield* expression would pick the value associated with done being true.
7. Introduce a Generator type that can be used to track the desired type of a yield expression. This would be TNext, and would be the type of the parameter for the next method on a generator.

The generator type would look something like this:

interface Generator<TYield, TReturn, TNext> extends IterableIterator<TYield> {
    next(value?: TNext): IteratorYieldResult<TYield> | IteratorReturnResult<TReturn>;
    throw(exception: any): IteratorYieldResult<TYield> | IteratorReturnResult<TReturn>;
    return(value: any): IteratorYieldResult<TYield> | IteratorReturnResult<TReturn>;
    [Symbol.iterator](): Generator<TYield, TReturn, TNext>;
    [Symbol.toStringTag]: string;
}

[ahejlsberg]

Boolean literal types are now available in #9407. Once that is merged we can update the return type of generators and iterators.

[s-panferov]

Does it allow to type async-like yield expressions in libraries like co or react-saga?

// getUser(): Promise<User>
let user = yield getUser()
// user: ?

[yortus]

@s-panferov unfortunately not. You are talking about the async-runner style of generator used by things like co right?

The generator proposal (#2873) doesn't offer much typing support for async-runner generators. In particular:

• All yield expressions are typed as any, as you can see in the example comments below. This is actually a very complex problem and I tried tackling it with some ideas which are all there in #2873. This won't change with #9407. And the TNext type in the OP above won't solve this either, since in an async runner there is generally not a single TNext type, but rather the type of each yield expression is a function of the type of that yield's operand (eg Promise<T> maps to T, Array<Promise<T>> maps to T[], etc). In particular the type of each yield expression is generally unrelated to the types of the other yield expressions in the body in an async-runner generator function.
• The compiler infers a single TYield type that must be the best common type among all the yield operands. For async runners there often is no such best common type, so such generators often won't compile until at least one yield operand is cast to any. E.g. the function *bar example below doesn't compile for this reason.
• Return types of generators are currently not tracked because without boolean literals, they can't be separated from the TYield type. This is what will be solvable once #9407 lands.

Example code:

interface User {id; name; address}
interface Order {id; date; items; supplierId}
interface Supplier {id; name; phone}
declare function getUser(id: number): Promise<User>;
declare function getOrders(user: User): Promise<Order[]>;
declare function getSupplier(id: number): Promise<Supplier>;

function* foo() {
    let user = yield getUser(42); // user is of type 'any'
    let user2 = <User> user;
    return user2; // This return type is not preserved
}

function* bar() { // ERROR: No best common type exists among yield expressions
    let user = yield getUser(42);       // user has type 'any'
    let orders = yield getOrders(user); // orders has type 'any'

    let orders2 = <Order[]> orders;
    let suppliers = yield orders2.map(o => getSupplier(o.supplierId)); // suppliers has type 'any'

    let suppliers2 = <Supplier[]> suppliers;
    return suppliers2; // This return type is not preserved
}

[s-panferov]

@yortus big thanks for the clarification!

All yield expressions are typed as any, as you can see in the example comments below. This is actually a very complex problem and I tried tackling it with some ideas which are all there in #2873. This won't change with #9407. And the TNext type in the OP above won't solve this either, since in an async runner there is generally not a single TNext type, but rather the type of each yield expression is a function of the type of that yield's operand (eg Promise maps to T, Promise[] maps to Promise<T[]>, etc). In particular the type of each yield expression is generally unrelated to the types of the other yield expressions in the body in an async-runner generator function.

Do you know if there is a tracking issue for this use-case? I think we definitely need to continue discussion, because this use-case is quite common and becomes more and more popular.

[yortus]

@s-panferov no problem. I think there's just #2873. There's quite a lot of discussion about the async-runner use-case in there, but I think that the team wanted to focus on getting simpler use cases working initially. Since that issue is now closed, I guess you could open a new issue focused specifically on better typing for co-style generators.

[DanielRosenwasser]

This hasn't actually been fixed yet.

[DanielRosenwasser]

The issue as I see it is that without #2175, this would be a breaking change. For example, you start out fixing IteratorResult:

interface IteratorYieldResult<Y> {
    done: false;
    value: Y;
}

interface IteratorReturnResult<R> {
    done: true;
    value: R;
}

type IteratorResult<Y, R> = IteratorYieldResult<Y> | IteratorReturnResult<R>

Now all of a sudden you need to introduce another type parameter to Iterator:

interface Iterator<Y, R> {
    next(value?: any): IteratorResult<Y, R>;
    return?(value?: any): IteratorResult<Y, R>;
    throw?(e?: any): IteratorResult<Y, R>;
}

which infects Iterable & IterableIterator:

interface Iterable<Y, R> {
    [Symbol.iterator](): Iterator<Y, R>;
}

interface IterableIterator<Y, R> extends Iterator<Y, R> {
    [Symbol.iterator](): IterableIterator<Y, R>;
}

These now break any users of Iterators. For instance, Array's members needed to be fixed up to:

interface Array<T> {
    /** Iterator */
    [Symbol.iterator](): IterableIterator<T, undefined>;

    /** 
      * Returns an array of key, value pairs for every entry in the array
      */
    entries(): IterableIterator<[number, T], undefined>;

    /** 
      * Returns an list of keys in the array
      */
    keys(): IterableIterator<number, undefined>;

    /** 
      * Returns an list of values in the array
      */
    values(): IterableIterator<T, undefined>;
}

[JsonFreeman]

Yes I remember our long discussion about this. The tricky bit is that many users will just want to use for-of, spread and rest, which never use the R type. Those users will not care about R, only Y. Then there are some users who will call the iterator methods explicitly, and they will care about the R type. The art is in serving both use cases simultaneously. I think there needs to be a type with two type parameters, and another type with only one, where the second type argument is any.

[falsandtru]

I feel definitions using literal types is too complex for common interfaces because we need to explicitly assert a boolean literal type for now. We need more easy ways to use literal types.

function iter(): IteratorResult<void, void> {
  return {
    done: <true>true
  };
}

[Igorbek]

With respect to what @JsonFreeman said according to the concern raised by @DanielRosenwasser, I experimented with a hypothetical typing of iterators that may return values.

Currently we have this:

interface IteratorResult<T> {
    done: boolean;
    value: T;
}

interface Iterator<T> {
    next(value?: any): IteratorResult<T>;
    return?(value?: any): IteratorResult<T>;
    throw?(e?: any): IteratorResult<T>;
}

This can be changed to: Sorry for names end with 2, that's just for illustration

interface IteratorYieldResult<Y> {
    done: false;
    value: Y;
}
interface IteratorReturnResult<R> {
    done: true;
    value: R;
}

type IteratorResult2<T, R> = IteratorYieldResult<T> | IteratorReturnResult<R>;
// redefine IteratorResult through extended interface to preserve generic arity
type IteratorResult<T> = IteratorResult2<T, any>;

interface Iterator2<T, R, I> {
    next(value?: I): IteratorResult2<T, R>;
    return?(value: R): IteratorResult2<T, R>;
    throw?(e?: any): IteratorResult2<T, R>;
}
// redefine Iterator through extended interface to preserve generic arity
type Iterator<T> = Iterator2<T, any, any>;

Open questions:

• next(value: I) or next(value?: I)
  • if value is optional then we will not be able to enforce strict type I in yield expression in generators :-1:
  • if value were not be optional then to allow next() we'd need to either type I | undefined or require consumer to work around like next(undefined as I) :-1:
• return(value: R) strictly requires value to be passed, right?

[JsonFreeman]

Nice typing @Igorbek!

I don't think the return parameter should be required. I get the impression that the return method is largely for cleanup, just executing any finally clauses that correspond to try/catch blocks surrounding the execution point (for a generator in particular). Unless you have a yield expression in one of the finally blocks, the consumer will already have the value they want returned.

For the next parameter, I think it has to be optional. Consider the language constructs for iterating (for-of, spread, etc). None of those constructs pass a parameter. If the generator really needs the parameter to be passed, then I think it is misleading to call it an iterator. It might be better to have a separate generator type for that, since the consumer will have to interact with the object in a different way.

[yortus]

Further to @JsonFreeman's comment, there are two very different uses generators in real-world code:

(1) For creating a series of values to iterate over:

• don't care about the return value
• don't pass anything to next
• do care about yielded values, which are usually all the same type

(2) For async runners (e.g. using the co library):

• do care about the return value, which is treated differently to the yielded values
• do pass values to next
• do care about yielded values, which are generally of unrelated types.

The latter case (async runners) should diminish with the growing awareness of async/await, but there's still a lot of existing code out there using generators this way.

[jesseschalken]

For reference, this is what Flow currently has for typing ES6 iterators and generators (from here, blog post here.). Flow is stricter than TypeScript but I suspect they had to make much of the same decisions.

type IteratorResult<Yield,Return> = {
  done: true,
  value?: Return,
} | {
  done: false,
  value: Yield,
};

interface $Iterator<+Yield,+Return,-Next> {
    @@iterator(): $Iterator<Yield,Return,Next>;
    next(value?: Next): IteratorResult<Yield,Return>;
}
type Iterator<+T> = $Iterator<T,void,void>;

interface $Iterable<+Yield,+Return,-Next> {
    @@iterator(): $Iterator<Yield,Return,Next>;
}
type Iterable<+T> = $Iterable<T,void,void>;

declare function $iterate<T>(p: Iterable<T>): T;

/* Generators */
interface Generator<+Yield,+Return,-Next> {
    @@iterator(): $Iterator<Yield,Return,Next>;
    next(value?: Next): IteratorResult<Yield,Return>;
    return<R>(value: R): { done: true, value: R };
    throw(error?: any): IteratorResult<Yield,Return>;
}

I don't think the return parameter should be required. I get the impression that the return method is largely for cleanup, just executing any finally clauses that correspond to try/catch blocks surrounding the execution point (for a generator in particular). Unless you have a yield expression in one of the finally blocks, the consumer will already have the value they want returned.

Unless the generator yields or returns inside a finally block as you said, .return(x) on a generator always returns {done: true, value: x}. So Flow appears to have the correct type with return<R>(value: R): {done: true, value: R}, although if you wanted to handle the case of yield or return in finally correctly it would be return<R super Return>(value: R): IteratorResult<Yield,R>.

I think calling .return() should infer R as undefined so you know you're going to get {done: true, value: undefined}.

For the next parameter, I think it has to be optional. Consider the language constructs for iterating (for-of, spread, etc). None of those constructs pass a parameter. If the generator really needs the parameter to be passed, then I think it is misleading to call it an iterator. It might be better to have a separate generator type for that, since the consumer will have to interact with the object in a different way.

Any generator that can be used with for..of must be prepared to see undefined as a result of yield, and IMO must include undefined in its input type (eg void), which will also allow calling .next() without a parameter. for..of on a generator that does not include undefined in its input type should be an error.

The reason Flow has the parameter to .next() as optional is because a generator first needs a call to .next() to start it, and the parameter provided to that call is ignored, but you can't express that a parameter is only optional for the first call.

[Igorbek]

@jesseschalken thanks a lot for the reference how Flow typed iterator/generator. There're a few things to consider or think about:

• return<R> - I personally don't like it, because it opens a way for consumer to cheat the contract. technically, a generator is not obligated to return exact value passed to return, it even can prevent closing and return done=false.
• they account for type variance :+1: ping #1394 #10717

As more I think about next's parameter then more I'm in favor of making it required. Of course, for..of calls it without any parameter, which in fact means argument's type would be undefined. But what if I want to use push side of a generator and expect to get something pushed in? Of course, I would not be able to iterate it with for..of, and that's ok. If I wanted I would have added undefined to the domain of next's parameter type. So that for-of-able is an iterator which has undefined in the domain of I generic type.

function* a(): Iterator2<number, string, string> {
  const s = yield 1; // s is string, not string|undefined
  return s;
}
for (let i of a()) {
              ~~~ Iterator2<number, string, string> cannot be used in for..of
}

function* b(): Iterator2<number, string, string|undefined> {
  const s = yield 1; // s is string|undefined
  return s || "";
}
for (let i of b()) { // ok
}

[JsonFreeman]

@Igorbek what about the first push, where you need not push anything?

[jesseschalken]

return<R> - I personally don't like it, because it opens a way for consumer to cheat the contract. technically, a generator is not obligated to return exact value passed to return, it even can prevent closing and return done=false.

Yep, as I said:

if you wanted to handle the case of yield or return in finally correctly it would be return<R super Return>(value: R): IteratorResult<Yield,R>.

This would allow you to call .return(), which would fill R with Return|undefined causing the result for that call to be {done: false, value: Yield} | {done: true, value: Return|undefined}, covering all three cases of yield, return and neither in the finally block.

However, last I checked TypeScript didn't have super type constraints, so I'm not sure how to otherwise express that in TypeScript.

edit: You could just do return<R>(value: R): IteratorResult<Yield,Return|R>

they account for type variance :+1: ping #1394 #10717

Yep, TypeScript's function parameter bivariance is a huge pain and we're considering migrating to Flow soon for that reason among other strictness benefits.

what about the first push, where you need not push anything?

The parameter to next() should probably be optional for that reason, but for..of should nonetheless still demand an iterator that accepts undefined as input.

edit: It occurred to me that because the typing is structural and the input type for the generator is only mentioned as the parameter to next, if that parameter is optional then a generator which doesn't include undefined in its input type will be indistinguishable from one that does.

[Igorbek]

@Igorbek what about the first push, where you need not push anything?

(I've been thinking so long) 😄

there's a stage-2 proposal that addresses the inconsistency in ignorance of the first pushed argument https://github.com/allenwb/ESideas/blob/master/Generator%20metaproperty.md

So if I want to rely on that data and enforce consumer to pass it, I would not be able to without making it required. The less restrictive case would be still achievable by | undefined.

[JsonFreeman]

I see. That sounds reasonable. But for iterators, it's still optional, right?

[Igorbek]

Yes, since next(value?: T) and next(value: T|undefined) are compatible.

[felixfbecker]

What is the status of this? As described in https://github.com/Microsoft/TypeScript/issues/11375, the current status quo makes Iterators pretty unusable, and it feels like it is unneccasserily blocked on trying to fix Generators in the same go. Can't we have separate IteratorResult and GeneratorResult types?

[StevenDoesStuffs]

Progress anybody?

[eddking]

It occurs to me that with generator return types you could achieve a modicum of type safety when using async runners. Instead of yielding values directly, we can wrap them in helper generator functions that return the result of a single yield and use them with yield *, the helper functions can enforce that the return type is appropriate for it's input.

Async await does not fully replace co runners like co & redux-saga, there are huge benefits to not performing side effects yourself. Essentially you can avoid the necessity of dependency injection, mocking and stubbing for testing and instead write code more naturally because at test time, you can simulate the side effects. This feature is still very important for us

[ronzeidman]

Joining in. I've replaced my async/await functions with /yield mostly since I needed cancellation support. I suspect many will do that once they realize that /yield with a co-routine gives a much better control on async execution and that is needed in mostly large projects that benefit from typescript the most.

Isn't it possible, at least for starters to just implement no-implicit-any flag support so this will fail:

function* itt(): IterableIterator<string> {
    const test = yield 'test';
    // test is implicitly any and there is nothing to do about it 
    //and there is no error telling me that test is any even if the no implicit any flag is on
}

Looking forward on seeing progress in this area.

[skishore]

I can see that making IterableIterator a type of arity 2 is going to cause a lot of trouble throughout the rest of the ES2015 typings. However, is it be possible to create an alternate Generator type with both Yield and Return types that can be explicitly annotated? Type inference for a generator would always infer Generator<T,any>, and Generator<T,U> would extend IterableIterator. That way, most of the library typings can stay unchanged.

[Jessidhia]

We have template defaults now, IterableIterator could become arity 2 without breaking existing code by giving the second argument an appropriate default.

[skishore]

Ah, you're absolutely right, and as eddking noted above, having types for the Yield and Return values is sufficient for many cases. The Next value mapping (e.g. co's Promise<T> -> T) can be written in a type-safe way as:

declare const getUser: (id: number) => Promise<User>;
...

const wait: <T>(promise: Promise<T>): IterableIterator<Promise<T>,T> => {
  return <T>(yield promise);
}

const my_async_method = co.wrap(() => {
  const user = yield* wait(getUser(42));
  ...
});

This is slightly more awkward than just const user = yield getUser(42);, but I think it's worth it. Is someone already working on adding the return type to IterableIterator? If not, I could take a look. I'm working on something similar to this async-runner use case that could really benefit from the additional safety!

[raveclassic]

Any considerations on this? Arity-2 IterableIterator has been mentioned several times to break things. But now we have default generic params and it seems like this can now be solved, doesn't it?

In addition to co and redux-saga I can mention fantasydo and its specialized version in Fluture. With latest TS it's now finally possible to fake HKT (monads in special) and such simulation of do-notation using generators would be extremely useful.

[shaunc]

So long as you are adding optional parameters, isn't there a third type -- the type that can be sent in to a generator == value passed to next() (which defaults to undefined?)

[JsonFreeman]

@shaunc that is TNext

[skishore]

Started looking into this. The type "IteratorResult" appears both in src/lib.es2015.iterable.d.ts and in the @types node module in node/index.d.ts. That causes an error when I fix the type in the src folder. Why is the type duplicated between those two places? I guess I'm going to need to update the node module and then up the version I have TypeScript depend on?

[skishore]

Never mind - the issue is quite clearly explained here: https://github.com/DefinitelyTyped/DefinitelyTyped/issues/17184

I have a clean way around that problem (remove the forward-declaration of IteratorResult; drop the next method from the forward-declaration of Iterator) which I'll submit as a patch to DefinitelyTyped. In the meantime, I am making progress on the TypeScript checker changes.

[skishore]

I ran into a significant roadblock. My implementation breaks the example below:

==== tests/cases/conformance/es6/destructuring/iterableArrayPattern1.ts (1 errors) ====
    class SymbolIterator {
        next() {
            return {
                value: Symbol(),
                done: false
            };
        }

        [Symbol.iterator]() {
            return this;
        }
    }

    var [a, b] = new SymbolIterator;
        ~~~~~~
!!! error TS2322: Type 'SymbolIterator' is not assignable to type 'Iterable<symbol, any, any>'.
!!! error TS2322:   Types of property '[Symbol.iterator]' are incompatible.
!!! error TS2322:     Type '() => SymbolIterator' is not assignable to type '() => Iterator<symbol, any, any>'.
!!! error TS2322:       Type 'SymbolIterator' is not assignable to type 'Iterator<symbol, any, any>'.
!!! error TS2322:         Types of property 'next' are incompatible.
!!! error TS2322:           Type '() => { value: symbol; done: boolean; }' is not assignable to type '(value?: any) => IteratorResult<symbol, any>'.
!!! error TS2322:             Type '{ value: symbol; done: boolean; }' is not assignable to type 'IteratorResult<symbol, any>'.
!!! error TS2322:               Type '{ value: symbol; done: boolean; }' is not assignable to type '{ done: true; value: any; }'.
!!! error TS2322:                 Types of property 'done' are incompatible.
!!! error TS2322:                   Type 'boolean' is not assignable to type 'true'.

Here, type IteratorResult<T, U = any> = {done: false, value: T} | {done: true, value: U}

Now, there are essentially two things going on that cause this issue. First off, the done field of the return type of the next method is always of type false, but due to type widening, it's inferred as type boolean. Second, TypeScript's current type assignment check cannot assign {done: boolean, value: number} to IteratorResult<number, any>, because a non-union type T can only be assigned to A | B if it can be assigned to A or it can be assigned to B. If you do the casework, however, this assignment should be legal.

I'm not sure which of these points is the best place to attack this problem. It's plausible that I could make the assignment check smart enough to handle this case. What do people think?

Edit: Perfect type inference is probably PSPACE-complete! It might only be NP-complete if TypeScript always organizes union and intersection types a certain way, though.

Also, if anyone wants to see what I've got so far, it's here: https://github.com/skishore/TypeScript/commit/668ff12ff75c6816942fb874816f91287622ca2c It includes the changes to the actual type signatures and the fix for checking the value type of an iterator, but no support for return and next type checks yet.

[brandonbloom]

I'm glad to see the above discussion. In the same vein, I encountered an area where the type checker seems overly permissive:

let f = function* (): IterableIterator<number> {
    yield 1;
    return 'str'; // no error?
}

If you omit the return type specification, then the inferencer produces the correct type for f:

() => IterableIterator<number | string>

[skishore]

() => IterableIterator<number | string> is still too loose a type for that function, though. The type parameters for the yield and return values should be distinct, since they're used in different places.

We haven't had much action on this thread for a while. Could someone on the Typescript team confirm whether my proposed change to the assignability condition is acceptable? I will finish my implementation if so.

[TazmanianD]

I'd like to recommend something I recently discovered that makes this whole problem moot (for Node at least). I've discovered node fibers which is sort of a replacement for generators (actually they came first). It's a native implementation that works (at least in appearance) very similar to generators. But the relevant usefulness is that the async functions look like normal functions. No need for IterableIterator which means that TypeScript can treat the function like a normal synchronous function. Also, fibers result in complete stack traces that follow the whole sequence of events which is awesome. So I would recommend taking a look at node-fibers and one of the modules built on top of it like f-promise.

Unfortunately this only applies if you're using Node. Fibers don't work anywhere else, including in a web browser.

[raveclassic]

Any updates on this? It's worth mentioning do-notation/for-comprehension can be actually faked quite easily: (example is based on fp-ts)

import { right, either, left } from 'fp-ts/lib/Either'
import { HKTAs, HKT2As, HKT2, HKTS, HKT, HKT2S } from 'fp-ts/lib/HKT'
import { Monad } from 'fp-ts/lib/Monad'
import { option, some } from 'fp-ts/lib/Option'

function Do<M extends HKT2S>(m: Monad<M>): <L, A>(generator: () => Iterator<HKT2<M, L, A>>) => HKT2As<M, L, A>
function Do<M extends HKTS>(m: Monad<M>): <A>(generator: () => Iterator<HKT<M, A>>) => HKTAs<M, A>
function Do<M extends HKTS>(m: Monad<M>): <A>(generator: () => Iterator<HKT<M, A>>) => HKT<M, A> {
  return <A>(generator: () => Iterator<HKT<M, A>>): HKT<M, A> => {
    const iterator = generator()
    const state = iterator.next()

    const run = (state: IteratorResult<HKT<M, A>>): HKT<M, A> => {
      if (state.done) {
        // any - if iterator is done, then its type is A, not HKT<M, A>
        return m.of(state.value as any)
      }

      return m.chain(value => run(iterator.next(value)), state.value)
    }

    return run(state)
  }
}

const res1 = Do(option)(function*() {
  const one = yield some(1) // any
  const two = 2
  const three = yield some(3) // any
  return one + two + three
})

const res2 = Do(either)(function*() {
  const one = yield right(1) // any
  const two = 2
  const three = yield left('Failure!') // any
  return one + two + three
})

console.log(res1) // some(6)
console.log(res2) // left("Failure!")

Type-checking yields/returns would be incredibly helpful when using monads.

[treybrisbane]

Any updates on this? I have the exact same usecase as presented above by @raveclassic

[treybrisbane]

I've put some thought into this, and I think the problem can be broken into two parts:

1. The ability to correctly type a generator function by describing the mapping between the types of the values passed to a generator's next method and the next method's corresponding return-types (or to think about it another way, the mapping between the types of the RHSs of a yield expression and the yield expression's resulting types)
2. The ability to ensure type-safety when executing coroutines with a task-runner (i.e. co)

A solution to the first part could be to describe the aforementioned mapping using a tuple-type whose elements are mapping's "entries" . E.g.

const c: () => Coroutine<[[undefined, Promise<number>], [number, Promise<string>], [string, string]]> = function* () {
  const aNumber = yield asyncAddTwo(4); // asyncAddTwo is of type `(n: number) => Promise<number>`
  const aString = yield asyncToString(aNumber * 2); // asyncToString is of type `(n: number) => Promise<string>`

  return `Result was ${aString}`;
}

Note that the "input" type in the first "entry" is undefined, since the first value we pass to next is undefined.

While the solution to the first part seems like it could be pretty straightforward, a solution to the second part is a bit harder.

Because of the way the ES6 generator interface was designed, I think what will ultimately be required is the ability for an object to type-guard itself as a result of one of its methods being invoked. In other words, we need a way of saying "After invoking method foo (which may return some type T) on object o of type O, object o should be considered to be some new type ONext.

Taking that idea, we could start describing the "steps" of our coroutine like this:

interface Coroutine<StepEntries extends [[any, any], [any, any]]> {
  next(value: StepEntries[0][0]): { done: false, value: StepEntries[0][1] } & this is CoroutineLast<[StepEntries[1]]>; // Note the `this` type-guard in addition to the actual method result

  // Similarly for `return` and `throw`
}

interface CoroutineLast<StepEntries extends [[any, any]]> {
  next(value: StepEntries[0][0]): { done: true, value: StepEntries[0][1] } & this is CoroutineLast<[[never, never]]>; // Coroutine is finished; further invocations make no sense

  // Similarly for `return` and `throw`
}

Going further, we could even generalise the Coroutine type to work for entry-tuples of arbitrary length:

type Inc = { [i: number]: number, 0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6, 6: 7, 7: 8, 8: 9, 9: 10 };

type TupleHasIndex<Tuple extends any[], I extends number> = ({[K in keyof Tuple]: 'T' } & 'F'[])[I];

type CoroutineResult<IsDone extends boolean, T> = { done: IsDone, value: T };
type Coroutine<StepEntries extends [any, any][], CurrentEntry extends [any, any] = StepEntries[0], I extends number = 1> =
  { T: { next(value: CurrentEntry[0]): CoroutineResult<false, CurrentEntry[1]> & this is Coroutine<StepEntries, StepEntries[I], Inc[I]> }, F: { next(value: CurrentEntry[0]): CoroutineResult<true, CurrentEntry[1]> } }[TupleHasIndex<StepEntries, I>];

Which could be used like so:

function runner<T1, R>(c: Coroutine<[[undefined, Promise<T1>], [T1, R]]>): Promise<R>;
function runner<T1, T2, R>(c: Coroutine<[[undefined, Promise<T1>], [T1, Promise<T2>], [T2, R]]>): Promise<R>;
// ...etc

[EliSnow]

Is there an issue with changing the interfaces in "lib.es2015.iterable.d.ts" to:

interface Iterator<T, N=any, R=any> {
    next(value?: N): IteratorResult<T>;
    return?(value?: R): IteratorResult<T>;
    throw?(e?: any): IteratorResult<T>;
}

interface Iterable<T, N=any, R=any> {
    [Symbol.iterator](): Iterator<T, N, R>;
}

interface IterableIterator<T, N=any, R=any> extends Iterator<T, N, R> {
    [Symbol.iterator](): IterableIterator<T, N, R>;
}

and similarly the interfaces in "lib.esnext.asynciterable.d.ts" to:

interface AsyncIterator<T, N=any, R=any> {
    next(value?: N): Promise<IteratorResult<T>>;
    return?(value?: R): Promise<IteratorResult<T>>;
    throw?(e?: any): Promise<IteratorResult<T>>;
}

interface AsyncIterable<T, N=any, R=any> {
    [Symbol.asyncIterator](): AsyncIterator<T, N, R>;
}

interface AsyncIterableIterator<T, N=any, R=any> extends AsyncIterator<T, N, R> {
    [Symbol.asyncIterator](): AsyncIterableIterator<T, N, R>;
}

?

[EliSnow]

(In partial answer to my above question), I was just wanting to get proper types for the next method, which the above amended interfaces give, but additional work is necessary for a generator to get the proper type for a yield statement.

[falsandtru]

Some actual usecases:

export class Coroutine<T, S = void> extends Promise<T> implements AsyncIterable<S> {
  constructor(
    gen: (this: Coroutine<T, S>) => Iterator<T | S> | AsyncIterator<T | S>,

https://github.com/falsandtru/spica/blob/master/src/coroutine.ts

class Cofetch extends Coroutine<XMLHttpRequest, ProgressEvent> {
  constructor(
    url: string,
    opts: CofetchOptions = {},
  ) {
    super(async function* (this: Cofetch) {
      this[Coroutine.destructor] = this.cancel;
      const xhr = new XMLHttpRequest();
      const state = new Cancellation<ProgressEvent>();
      const process = new Colistener<ProgressEvent, XMLHttpRequest>(listener => {
        void xhr.addEventListener('loadstart', listener);
        void xhr.addEventListener('progress', listener);
        void xhr.addEventListener('loadend', listener);
        void ['error', 'abort', 'timeout']
          .forEach(type =>
            void xhr.addEventListener(type, state.cancel));
        void fetch(xhr, url, opts);
        void this.cancellation.register(() =>
          xhr.readyState < 4 &&
          void xhr.abort());
        return () => undefined;
      });
      for await (const ev of process) {
        assert(ev instanceof ProgressEvent);
        assert(['loadstart', 'progress', 'loadend'].includes(ev.type));
        yield ev;
        if (ev.type !== 'loadend') continue;
        void state.either(xhr)
          .extract(
            process[Coroutine.terminator],
            process.close);
      }
      return process;
    }, {}, false);
    void this[Coroutine.run]();
  }
  private readonly cancellation = new Cancellation();
  public readonly cancel: () => void = this.cancellation.cancel;
}

https://github.com/falsandtru/spica/blob/master/src/cofetch.ts

    it('basic', async () => {
      const co = cofetch('');
      const types = new Set<string>();
      for await (const ev of co) {
        assert(ev instanceof ProgressEvent);
        assert(['loadstart', 'progress', 'loadend'].includes(ev.type));
        types.add(ev.type);
        if (ev.type !== 'loadend') continue;
        for await (const _ of co) throw 1;
      }
      assert.deepStrictEqual([...types], ['loadstart', 'progress', 'loadend']);
      assert(await co instanceof XMLHttpRequest);
    });

https://github.com/falsandtru/spica/blob/master/src/cofetch.test.ts

The yield values are observable (async iterable) part, return value is promise part.

[jonaskello]

Being able to model the return type for iterators does not seem to be on the roadmap at all. This seems a bit strange to me since generators are a core part of the language that typescript currently cannot model correctly. To me, being able to correctly model core language features seems like it should have some priority. I know this is a hard problem to solve, but perhaps it at least should be added to the roadmap for future investigation?

[treybrisbane]

Strongly agree that this should be on the roadmap.

[VictorQueiroz]

We need this!

[lukasbash]

Not sure if a can gather some real decision from the very long discussion above. Any update on this? I think generators are not niche anymore. I strongly believe that it is necessary to fully model them and their features. Libraries that heavily rely on this feature would be even more useful. I am using redux-saga and doing all this manual type definitions just clutters the code imho.

You guys are doing a great job, devs really need this feature!

[robbyemmert]

This is a critical feature for me as well. Generators are here to stay.

Any workarounds with custom types?

[falsandtru]

@DanielRosenwasser Can you please address this issue?

[Igorbek]

The issues in my previous typings with optionality of parameters of next and return can now be solved with conditional types.

interface GeneratorYieldResult<Y> {
    done: false;
    value: Y;
}

interface GeneratorReturnResult<R> {
    done: true;
    value: R;
}

type GeneratorResult<T, R = undefined> = GeneratorYieldResult<T> | GeneratorReturnResult<R>;

interface Generator<T, R = unknown | undefined, I = R | undefined> {
    readonly next: undefined extends I
        ? (value?: I) => GeneratorResult<T, R>
        : (value: I) => GeneratorResult<T, R>;
    readonly return: undefined extends R
        ? (value?: R) => GeneratorReturnResult<R>
        : (value: R) => GeneratorReturnResult<R>;
    throw(e?: any): GeneratorResult<T, R>;
}

With this stricter contract can be established:

declare const g1: Generator<number>;
g1.next();      // ok, optional
g1.next(1);     // ok, accepts unknown
g1.return();    // ok, optional
g1.return(1);   // ok, accepts unknown

declare const g2: Generator<number, string>;
g2.next();      // ok, optional
g2.next('');    // ok, accepts string | undefined
g2.return();    // error, required
g2.return('');  // ok, accepts string

declare const g3: Generator<number, string, boolean>;
g3.next();      // error, required
g3.next(true);  // ok, accepts boolean
g3.return();    // error, required
g3.return('');  // ok, accepts string

Having that we should be able to write:

function* g4(): Generator<number, string, boolean> {
    const firstValue /* infers boolean */ = function.sent; // ECMAScript Stage 2 proposal
    const secondValue /* infers boolean */ = (yield 1);
    return "result";
}

Also

function* g5() { // return type should be inferred as Generator<string, boolean, undefined>
  yield "a";
  return true;
}

[Igorbek]

Sorry, I forgot that termination caused by return can be prevented:

    readonly return: undefined extends R
-        ? (value?: R) => GeneratorReturnResult<R>
+        ? (value?: R) => GeneratorResult<T, R>
-        : (value: R) => GeneratorReturnResult<R>;
+        : (value: R) => GeneratorResult<T, R>;

[Alxandr]

This is really neat. But I would say an issue is that you can't describe (in types) the transformation that happens based on the yielded type though, no? Like; if you do await getPromise<T>(), typescript know that the output of that expression is of type T. But for generators, that depends. With conditional types though, there exist a way to actually type this:

type PromiseReturn<T> = T extends Promise<infer K> ? K : never;

Now, if we wanted to create promsies using a generator library, we could imagine typing this as something like this:

function runAsPromise<TRet>(gen: Generator<PromiseReturn, TRet>): Promise<TRet>;