Closed haberdashPI closed 4 years ago
Probably the simplest approach here is to just return a (potentially lazy) array chunk, that get's written to the result. Then there aren't separate atcheckpoint
and sampleat!
methods, just a nextchunk
method.
Having thought about this more, as stated above, this approach would not interact well with mapsignal
: the advantage of the two stage, atcheckpoint
and sampleat!
approach, is that we can merge the checkpoints across children.
A way of maintaining this advantage would be to have two stages: maxchunksize
and nextchunk
. The child reports what the longest result for the next size can be. Then a request is made for a chunk of that size or smaller. Both methods take as input the signal and the current offset, and nextchunk
also takes the requested chunk size.
It's still pretty confusing. I think there could be some generic merging code, and that would simplify the individual calls to
checkpoints
substantially.