Level / party

Open a leveldb handle multiple times.
MIT License
146 stars 13 forks source link

perf: bypass duplexify #29

Closed ronag closed 2 years ago

ronag commented 3 years ago

Bypasses duplexify and pipes encode, socket, decode directly without any intermediate proxy.

ronag commented 3 years ago

Would be nice if multileveldown provided a less hacky way to do this...

vweevers commented 3 years ago

The goal here is to skip this line, if I understand it correctly? https://github.com/Level/multileveldown/blob/52f4861fc37500bfed167114f007bfce3104fc77/leveldown.js#L79.

ronag commented 3 years ago

The goal here is to skip this line, if I understand it correctly? https://github.com/Level/multileveldown/blob/52f4861fc37500bfed167114f007bfce3104fc77/leveldown.js#L79.

Yep and avoid the overhead and complexity of an intermediate stream.

vweevers commented 3 years ago

The less hacky way would be to support a second argument here, defaulting to null:

https://github.com/Level/multileveldown/blob/52f4861fc37500bfed167114f007bfce3104fc77/client.js#L21-L23

ronag commented 3 years ago

The less hacky way would be to support a second argument here, defaulting to null:

Yep. But that would require updating multileveldown.

vweevers commented 3 years ago

I worry this will have a side effects, because:

ronag commented 3 years ago

I worry this will have a side effects, because:

  • duplexify handles a bunch of edge cases

I'm more worried about having duplexify than not having it.

  • the close event on a net socket has a hadError argument that we may need to ignore here

Yes.

vweevers commented 2 years ago

This makes no difference in my benchmarks, at least not after removing much bigger bottlenecks. In rave-level (the WIP replacement for level-party) I doubled the throughput by batching reads, among other things.

ronag commented 2 years ago

Can I follow many-level somewhere?

vweevers commented 2 years ago

I'll post something in https://github.com/Level/abstract-level/issues/14 once I've pushed the repo