library defined iterators

[timotheecour]

• experiment showing how iterators can (almost) be implemented with a library solution by timotheecour · Pull Request #15655 · nim-lang/Nim
• can iterators be implemented as a library solution / syntax sugar over regular procs? · Issue #272 · nim-lang/RFCs

would fix:

• Closure iterators do not respect try/finally or defer · Issue #15501 · nim-lang/Nim
• Closure iterators are not supported by VM by xflywind · Pull Request #15818 · nim-lang/Nim

TODO

check which other bugs would std/iterates fix; eg:

• js: lent doesn't work in iterators with tuple destructuring · Issue #14574 · nim-lang/Nim
• Tuple destructuring is broken with closure iterators · Issue #15924 · nim-lang/Nim
• compiler error if combine closure iterator, normal iterator and template · Issue #15896 · nim-lang/Nim

links

• cps/iterator.nim at master · disruptek/cps
• Gagallium : Generators, iterators, control and continuations
• partially related: Passing iterators as arguments - Nim forum

[HJarausch]

The handling of break is broken. Let's call this newStyleIterator. If a newStyleIterator is called within an newStyleIterator the generating exception should NOT be caught. Furthermore, this implementation doesn't allow for newStyleIterators with optional arguments or a variable number of arguments. In addition there is no miterate.

The attached modification has following differences:

1. it defines and uses StopIterate instead of IterateError
2. the generated call to the newStyleIterator uses the keyword parameter Yield
3. it defines 4 macros: iterate, miterate, riterate and rmiterate.

• iterate is only a slight modification of the original version, i.e. it wraps the call to the generator in try..except
• miterate just makes the iterator variable modifiable
• riterate must be called instead of iterate within an newStyleIterator - it doesn't catch StopIterate
• rmiterate is the variant of riterate with a modifiable iterator variable

iterates.nim.txt Perm.nim.txt

[timotheecour]

thanks, will look into it tmrw!

[HJarausch]

Finally, with the fantastic help from several people (especially on Gitter) I managed to fix your iterate macro. The idea is as follows:

the macro generates:

• a global counter StopIterateCount
• a break is transformed to raise newStopIterate(StpItCount) -- see below
• the body of the for loop is transformed into a

  inc(StopIterateCount)
  let StpItCount{.inject.} = StopIterateCount 
  try: .... 
  except StopIterate as EXC :
  if EXC.Counter == StpItCount :
  discard
  else :
   raise

  Now, an original break breaks only the loop where it was coded.

This work with recursive calls of the iterator to itself. The identifiers StopIterateCount, StopIterate and StpItCount are changed in the real implementation s.t. these cannot be used by accident. iterates.nim.txt Perm.nim.txt