Add ability to yield var's for iterators

[philip-wernersbach]

Iterators should be able to yield var's.

[Varriount]

In this context, how would a var return type be different from a regular return type?

[Araq]

But they do. Look at tables.mitems if you don't believe me.

[philip-wernersbach]

tables.m* works, but for some reason the following simple test case does not:

iterator testVarIterator(x: var seq[string]): var string =
    for s in x:
        var sv = s
        yield sv

var foo = @["Hello", "World"]
for s in foo.testVarIterator():
    echo s

var_iterators_test.nim(4, 15) Error: address of 'sv' may not escape its stack frame

Weird.

[Araq]

Not hard to fix but I'm unsure wether it's wrong. Why do you need to do this?

[philip-wernersbach]

I need it to avoid unnecessary things like this in nim-orient. Most of the API for using a OrientRecord requires var OrientRecord's, so nothing meaningful can be done with a non-var OrientRecord. In this case, I tried to yield a var OrientRecord from here, but I got the previous error.

[Araq]

Perhaps you should implement OrientRecord as a ref object or ptr object then?

[philip-wernersbach]

That's probably a better option. What would the performance implications of that? I assume a ref hits the GC more than a var.

[Varriount]

As far as I can tell, a var type is a type always either on the stack by itself, or on the heap as an item in a collection. A vat type is possess via a pointer to the variable entry in the C backend.

[mratsim]

Got hit by this while trying to port Arraymancer to Javascript, here is a test case:

var a = @[1, 2, 3, 4, 5]

# Current Arraymancer implementation
proc dataArray*[T](s: seq[T]): ptr UncheckedArray[T] {.inline.} =
  cast[ptr UncheckedArray[T]](s[0].unsafeAddr)

let u = a.dataArray
echo u[2] # 3 with C backend, undefined with JS backend

# Alternative implementation for JS
proc seqArray*[T](s: seq[T]): seq[T] {.inline.} =
  shallowCopy(result, s)

let s = a.seqArray
echo s[2] # 3 with C backend, 3 with JS backend

##################################################@
# Now, issue is in mitems

iterator mitems2[T](s: var seq[T]): var T =
  var data = s.seqArray

  for i in 0 ..< s.len:
    yield data[i] # Using s directly works,
                  # With data, we get "address of 'data[i]' may not escape its stack frame"

for v in a.mitems2:
  v *= 2

echo a

In reality, dataArray also add a constant offset like this

proc dataArray*[T](s: seq[T]): ptr UncheckedArray[T] {.inline.} =
  cast[ptr UncheckedArray[T]](s[CONSTANT_OFFSET].unsafeAddr)