search

mwotton / judy

Other
18 stars 4 forks source link

Strip out lazy IO #6

Closed chrisdone closed 6 years ago

chrisdone commented 6 years ago

Fixes #5.

mwotton commented 6 years ago

thanks @chrisdone , this looks great.

@ketil-malde could you check this with your dataset to see if it's the problem? it looks ok to me - it's possible that people are relying on laziness, but I don't think I know how to retain laziness robustly - would probably have to be some kind of more principled Conduit or Pipes thing.

ketil-malde commented 6 years ago

On 2017-12-19 14:57 (+0100), Mark Wotton notifications@github.com wrote:

thanks @chrisdone , this looks great.

@ketil-malde could you check this with your dataset to see if it's the problem? it looks ok to me - it's possible that people are relying on laziness, but I don't think I know how to retain laziness robustly - would probably have to be some kind of more principled Conduit or Pipes thing.

Yes, I worry about performance here, but if it does indeed get rid of the problems I've encountered, we could have some kind of folding IO operation instead (basically, I just need to write out the information from the Judy array)

Good catch, at any rate.

-k -- If I haven't seen further, it is by standing in the footprints of giants

chrisdone commented 6 years ago

I think providing some kind of iterator like a fold or a stream (pipe/conduit) would be the more modern approach, yeah.