Iterator Records as a user-facing data structure?

Should iterator records, _iteratorRecord, be a user-facing data structure? I am for this as it allows the user to overload them as they wish and it would make it more clear when the user expects to take an iterable as an argument.

In particular, overloading the _iteratorRecord can be rather fun... Some examples that you can do right now...

Operating on a Random Stream

use Random;

iter _iteratorRecord.map(fn) {
    for x in this do yield fn(x);
}

iter _iteratorRecord.filter(fn) {
    for x in this do if fn(x) then yield x;
}

proc _iteratorRecord.reduction(fn) {
    var x : fn.retType;
    for y in this do x = fn(x,y);
    return x;
}

proc _iteratorRecord.consume(fn) {
    for x in this do fn(x);
}

iter randomNumberGenerator(sampleSize) {
    var rng = makeRandomStream(int);
    for 1..sampleSize {
        yield rng.getNext();
    }
}

writeln(randomNumberGenerator(10)
    .map(lambda(x : int) : int { return x % 100; })
    .filter(lambda(x : int) : bool { return x % 2 == 0; })
    .reduction(lambda(x : int, y : int) : int { return max(x, y); })
);

TIO

Gathering by Categories

use List;

iter _iteratorRecord.map(fn) {
    for x in this do yield fn(x);
}

iter _iteratorRecord.filter(fn) {
    for x in this do if fn(x) then yield x;
}

iter _iteratorRecord.groupBy(fn) {
    var data = this;
    var categoryDomain : domain(fn.retType);
    var categories : [categoryDomain] list(data.eltType);
    for d in data {
        var category = fn(d);
        categoryDomain += category;
        categories[category].push_back(d);
    }
    for dom in categoryDomain {
        yield(dom, categories[dom]);
    }
}

proc _iteratorRecord.reduction(fn) {
    var x : fn.retType;
    for y in this do x = fn(x,y);
    return x;
}

proc _iteratorRecord.consume(fn) {
    for x in this do fn(x);
}

writeln((1..10).these().groupBy(lambda(x : int) : bool { return x % 2 == 0; }));

TIO

Some pretty cool stuff. The current issue is that currently it can't do parallel iteration, which would be something that could be provided on a user-facing version of the structure. With a combination of both this and first-class function support (I know, its no where near on your TODO list, but it is on mine), I think we could have some nice functional programming going on, or at least emulate something like Java 8 Streams which helps appeal to more types of users, yay! :smile:

Perhaps what could be done would be add on to the _iteratorRecord to create a user-facing version with the following API...

record Iterator {
   //… Internal State ...
}
proc Iterator.isStandalone();
proc Iterator.isLeader();
proc Iterator.isFollower();
proc Iterator.isSerial();
// … etc ...

Edit:

Another cool example of things you can do easily with this kind of overloading...

Infinite Streams of Random Numbers

use Random;

iter _iteratorRecord.map(fn) {
    for x in this do yield fn(x);
}

iter _iteratorRecord.filter(fn) {
    for x in this do if fn(x) then yield x;
}

proc _iteratorRecord.reduction(fn) {
    var x : fn.retType;
    for y in this do x = fn(x,y);
    return x;
}

proc _iteratorRecord.first(fn) {
    for x in this {
        if fn(x) then return x;
    }
    halt("'first' could not find anything");
}

iter infiniteRandomNumberGenerator() {
    var rng = makeRandomStream(int);
    while true {
        yield rng.getNext();
    }
}

writeln(infiniteRandomNumberGenerator()
    .map(lambda(x : int) : int { return x % 100; })
    .filter(lambda(x : int) : bool { return x % 2 == 0; })
    .first(lambda(x : int) : bool { return x >= 70; })
);

TIO

chapel-lang / chapel

Iterator Records as a user-facing data structure? #10361