search

flumedb / flumelog-offset

MIT License
10 stars 8 forks source link

Add method to stream without cache #10

Closed arj03 closed 6 years ago

arj03 commented 6 years ago

When doing large reads the overhead in a cache in the getMeta function can be quite substantial (15%). Also if I know that I'm going to do a lot of reads that I will not need again, there is no reason to blow the cache for everyone else.

I think it would be a good idea to use it here so that indexing is faster.

dominictarr commented 6 years ago

I think I'd rather have an option to disable cache on stream rather than a second method. It might also be good to use that option when, say, rebuilding views? I suggest cache option that defaults to false, so {cache: false} streams everything without cache.

arj03 commented 6 years ago

Yes, it would be really useful for rebuilding views. I'll look into the cache option.

arj03 commented 6 years ago

Thanks for pointing me in the right direction dominic. Adding it to pull-cursor is a more modular design.

arj03 commented 6 years ago

This should be good now I think. Depends on https://github.com/dominictarr/pull-cursor/pull/2.

arj03 commented 6 years ago

@dominictarr I merged this and bumped versions. The same with bench-flumelog 1.2.0 but I don't have permissions on npm to publish them. I did publish pull-cursor 3.0.0 though.

dominictarr commented 6 years ago

added you to flume-owners, now you have everything!

On 5/31/18, Anders Rune Jensen notifications@github.com wrote:

@dominictarr I merged this and bumped versions. The same with bench-flumelog 1.2.0 but I don't have permissions on npm to publish them. I did publish pull-cursor 3.0.0 though.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/flumedb/flumelog-offset/pull/10#issuecomment-393146615