[Feature 🔨]: Generator

[seiyab]

Is your feature request related to a problem? Please describe.

No, just an idea. It should be convenient because this pattern can be seen in other languages. examples: Python, JavaScript, Kotlin

Describe the solution you'd like

Creating iterator from imperative function.

Provide code snippets to show how this new feature might be used.

func ExampleGenerator() {
  it := iter.Generator[string](func(yield func(v string)) {
    yield("hello")
    yield("generator")
  })
  fmt.Println(iter.Collect(it))
  // Output: [hello generator]
}

Does this incur a breaking change?

no.

Do you intend to build this feature yourself?

Maybe. Feel free to comment if someone intend to commit.

Additional context I'm not sure whether it is good feature or not. It's because following reasons.

• It is imperative rather than functional.
• The functionality can be nearly achieved by #29 .

Justifications exist, but weak.

• It bridges gaps between imperative and functional.
• It is a bit clear and readable.

[seiyab]

function name can be FromProcedure, FromRoutine FromFunction, Go or something else

[dranikpg]

This is definitely an interesting feature.

Justifications exist, but weak.

Contrary to iterators from channels, this one would have no overhead and should be preferred if you don't work with concurrency/plug into existing code.

However, real generators require support up from the language level, which Go has not. You can't "suspend" the callstack like a stackless coroutine and return from a function multiple times.

This means that this would have to be implemented with a producer goroutine and channels. Which is not that far away from just using a iterator from a channel:

func ExampleGenerator() {
  it := iter.Generator[string](cs chan<- string) {
    cs <- "hello"
    cs <- "generator"
  })
  fmt.Println(iter.Collect(it))
  // Output: [hello generator]
}

which is doable with just:

func Generator[T any](f func(chan<- T)) *ChannelIter[T] {
  cs := make(chan T)
  go func() {
    f(cs)
    close(cs)
  }()
  return FromChannel[T](cs)
}

In case the iterator is dropped and not exhausted, the spawned goroutine will stay around forever 😕

[seiyab]

@dranikpg Thank you for your helpful comment.

In case the iterator is dropped and not exhausted, the spawned goroutine will stay around forever

That's true. So feasible options will be following, and each of them has clear disadvantages.

1. using chan T, chan any (for interrupt) and explicit Close method

   • func ExampleGenerator() {
       it := iter.Generator[string](func(yield func(v string) bool) {
         if !yield("hello") { return }
         if !yield("generator") { return }
       })
       defer it.Close()
       fmt.Println(iter.Collect(it))
       // Output: [hello generator]
     }

     • disadvantages
     • need explicit Close
     • the lifecycle might be complex when it is wrapped by higher-order iterators.
     • need to check the result of yield
     • it might be better that this functionality is provided by another package for utility of chan and then just wrap it by ChanIter.
2. using T[]

   • func Generator[T any](f func(yield func(v T)))*LiftIter[T] {
     var items []T
     yield := func(v T) {
       items = append(items, v)
     }
     f(yield)
     return Lift(items)
     }

   • disadvantages
     • non-lazy evaluation is unexpected and confuses developer
     • it cannot handle infinite iterator
3. in-place Next

   • func ExampleGenerator() {
     it := iter.Generator[int](func() func() int {
       var i int = 1
       return func() int {
         r := i
         i *= 2
         return r
       }
     })
     fmt.Println(iter.Collect(iter.Take(it, 3)))
     // Output: [1 2 4]
     }

   • disadavantage
     • the API is far from generator. it might be better to implement as plain custom Iterator as you mentioned first.

As far as I know, this suggestion can't be useful enough. I will close this issue if there is no good idea.

[dranikpg]

I'm afraid the won't be any more ideas 😢

The first option is not that bad actually, except for the Close. Having to close the iterator is inconvenient almost impossible, especially if you want to pass it around and wrap it up further.

The third option is usable. Instead of passing a function returning a closure, we could just directly pass the closure. If someone needs local state, he can just create this helper function on its own or store it directly in the function using the iterator.

func CreateMagicSequence(offset int) Iterator[int] {
  r := rand.New()
  return Generator[int](func(index int) Option[int] {
    return Option.Some(index * r.Intn(10) + offset)
  }
}

This still more compact than defining your own type + contructor + Next() function.

[BooleanCat]

I like this option too, where the Generator is a simple convenience for a stateless Iterator (or where state is captured through a closure). Unlike @dranikpg I think the Generator input signature should match that of Next() (without the receiver):

func Generator[T any](gen func() option.Option[T]) GeneratorIter { ... }

[seiyab]

Passing a closure directly sounds good. I prefer the signature that matches that of Next().

Perhaps the name Generator is confusing for current idea?

[BooleanCat]

Perhaps simply iter.New(f) *FuncIter { ... }

[BooleanCat]

Maybe hold fire on this until the Go iterator conversation comes to fruition.

https://github.com/golang/go/issues/61897

[BooleanCat]

Well Go iterators are a thing soon! I think an API like this would be appropriate:

func Generator[V any](fn func() V) iter.Seq[V] {
  ...
}

and

func Generator2[V, W any](fn func() (V, W)) iter.Seq2[V, W] {
  ...
}

Note that the current latest implementation is in the branch v2.

Not sure if you're still interested in working on this @seiyab ?

[seiyab]

Thank you for notifying me! Feel free to implement it ignoring me. I'm still interested in this but I'm busy for a while because of my life event. I might work on it when I will get time.

[BooleanCat]

I don't think this is needed any longer given the new iter.Seq syntax. Happy to revisit if there's a need for it from users.