Get the total number of elements in an iterator

[kjetilk]

I was thinking, it might be very good to be able to have the number of elements in an iterator in many cases, for example for join order optimization. Part of the problem with my cache is that the generated plans really fails at the join order optimization, even though the cardinality is known, and for one of the triple patterns is really small...

Perhaps have a role for it? An Attean::API::CountableIterator role. Or perhaps that isn't appropriate, since a CodeIterator may or may not be countable.

Anyway, wanted to air it... :-)

Kjetil

[kasei]

For this particular case, I'd suggest that you should be making join order decisions at planning time (assuming you have access to the count of results at planning time, too). CodeIterators would be very hard (and sometimes impossible) to provide counts for without fully consuming them and essentially turning them into ListIterators. If you really want this, I'd suggest you can add a size method to ListIterator and make sure you're using those wherever you want sizes.

[kasei]

In fact, I'm just going to commit a Attean::ListIterator->size method. I still think you should do this at planning time, but this will be available if you need it.

[kjetilk]

Yeah, so my main idea for this was Attean::Plan::Iterator, where the iterator is given at planning time. Bad idea?

[kasei]

Where is that iterator coming from, though? At some point, it's being created with knowledge of what's going into it. Is that not happening at planning time?

[kasei]

There are now size methods on both Attean::ListIterator and Attean::API::RepeatableIterator (though the latter is only a fallback at this point since there are no other consumers of RepeatableIterator beyond ListIterator). Closing this as resolved.