closeable iterators

[CeylonMigrationBot]

[@gavinking] For stuff like database cursors, it would be really nice if you could have your Iterator be Closeable, and have the for loop automatically call close() however iteration terminates (whether by return, throw, break or exhaustion).

I believe this can be implemented for ordinary for loops. Not so sure about comprehensions.

[Migrated from ceylon/ceylon-spec#895]

[CeylonMigrationBot]

[@lucaswerkmeister] Would entering the loop also call open(), or do you want to leave that to initialization code outside the loop?

Also, I see no problem in implementing this for comprehensions, as long as it’s the Iterator that’s closable (not the Iterable).

[CeylonMigrationBot]

[@gavinking] Well, there is a proposal in #4000 to introduce Enlistable and remove/deprecate Closeable.open(). So an Iterator could be Closeable. Not sure if it could make sense for it to be Enlistable.

[CeylonMigrationBot]

[@quintesse] Btw, Won't this give problems in those cases where things like filter() either use custom Iterators or use comprehensions (that internally also use iterators)? Because you won't be able to to do is Closeable to figure out if should be closed or not.

[CeylonMigrationBot]

[@gavinking] I don't understand. Every for would behave the same, even in a comprehension.

Sent from my iPhone

On 29 May 2014, at 1:40 pm, Tako Schotanus notifications@github.com wrote:

Btw, Won't this give problems in those cases where things like filter() either use custom Iterators or use comprehensions (that internally also use iterators)? Because you won't be able to to do is Closeable to figure out if should be closed or not.

— Reply to this email directly or view it on GitHub.

[CeylonMigrationBot]

[@quintesse] I mean something like this:

Iterable<String> foos = getIterableThatReturnsDestroyableIterators();
for (f in foos.filter(myFilterFunc)) {
    print(f);
    if (f == "bar") break;
}

How does this for know that it should destroy the iterator? In fact how can it destroy it? Because filter() has wrapped the original destroyable iterator in a new one to implement the filtering, and that new iterator isn't destroyable.

Unless we re-implement all those wrapping iterators to be destroyable by default and then do something like:

shared void destroy() {
    if (is Destroyable wrappedIterator) {
        wrappedIterator.destroy();
    }
}

Is that what you're thinking?

Edit: ah yes @lucaswerkmeister , the correct name is Destroyable, I fixed that in the above text and code. I'm not sure that all Iterators would implement it, if that's the case then it's okay, but the text of this issue suggests it's a choice, not a requirement.

[CeylonMigrationBot]

[@lucaswerkmeister] I thought all Iterators would implement Destroyable, with a default no-op implementation? Iterator extends Destroyable?

[CeylonMigrationBot]

[@tombentley] @quintesse I think that once #4000 is implemented writing a method such as getIterableThatReturnsDestroyableIterators() would be impossible, because it would be leaking a Destroyable, though @gavinking hasn't described how those rules would change to support destroyable iterators (presumably Iterable.iterator() would be allowed to leak a destroyable iterator).

[CeylonMigrationBot]

[@quintesse]

would be impossible

Well maybe, but the situation remains the same if it were:

for (f in IterableThatReturnsDestroyableIterators().filter(myFilterFunc))

right?

[CeylonMigrationBot]

[@gavinking] Why can't the wrapper Iterator be Destroyable and delegate close()? Still not seeing the problem.

[CeylonMigrationBot]

[@tombentley] Or maybe we're not seeing the solution... or rather how well this solution would apply to ceylon/ceylon-sdk#241. AFAICS because for requires an Iterable we can't make a Reader do Iterable-like things because (e.g. when you're reading from a network socket) the Reader can't supply an arbitrary number of Iterators (without keeping all the input in a buffer) -- it can only read from the socket once, so it can only supply one Iterator.

[CeylonMigrationBot]

[@gavinking] Eh? Once we have destroyable iterators what is the need for Reader?

[CeylonMigrationBot]

[@tombentley] No, my point is that Iterators on their own have no useful methods or language support the way Iterable does, and you can only read once.

[CeylonMigrationBot]

[@gavinking] But that doesn't matter. From the user perspective it is:

for (line in file.lines) { ... }

Where file.lines gives you a {String*}. The only reason Reader exists at all is because iterators can't be heavyweight.

[CeylonMigrationBot]

[@gavinking] Note that if we do this for for we should also consider doing it for let, perhaps. This would kinda dupe functionality of try but still worth considering.

[CeylonMigrationBot]

[@gavinking] I wonder again whether destruction could be a totally implicit thing that happens to any Destroyable when the scope it belongs to is destroyed. There would be some challenges with that idea, but it's not completely impossible AFAICT.

[CeylonMigrationBot]

[@quintesse]

Why can't the wrapper Iterator be Destroyable and delegate close()? Still not seeing the problem.

That's what I asked: "Unless we re-implement all those wrapping iterators to be destroyable by default and then [delegate]. Is that what you're thinking?"

[CeylonMigrationBot]

[@gavinking] Yes, right.