housepower / ClickHouse-Native-JDBC

ClickHouse Native Protocol JDBC implementation
https://housepower.github.io/ClickHouse-Native-JDBC/
Apache License 2.0
527 stars 145 forks source link

Further minimize buffer allocation in INSERT path. Remove one of writeBinary() overrides. #396

Closed AndreyNudko closed 2 years ago

AndreyNudko commented 2 years ago

Instead of copying serialized column data from ByteBuffer via intermediate byte[], just write backing array directly. This somewhat couples ColumnWriterBuffer with ByteArrayWriter internals, however as the name implies ByteArrayWriter is expected to use heap buffers (and using direct buffers for block data wouldn't make much sense now anyway - I/O layer is operating on byte[], plus most JVMs impose limits on direct memory allocation). ByteArrayWriter and CompressedBuffedWriter use fixed-size buffers internally and chunk bytes already. SocketBuffedWriter may benefit from splitting large writes into smaller ones to minimize malloc/free in native code - but it shouldn't be worse than current situation (at least with compression) and likely to be dubious optimisation.

codecov[bot] commented 2 years ago

Codecov Report

Merging #396 (14eff7a) into master (2234592) will decrease coverage by 0.06%. The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #396      +/-   ##
============================================
- Coverage     62.43%   62.37%   -0.07%     
+ Complexity     1250     1246       -4     
============================================
  Files           135      135              
  Lines          6640     6628      -12     
  Branches        516      514       -2     
============================================
- Hits           4146     4134      -12     
  Misses         2219     2219              
  Partials        275      275              
Impacted Files Coverage Δ
.../com/github/housepower/buffer/ByteArrayWriter.java 100.00% <ø> (ø)
...thub/housepower/buffer/CompressedBuffedWriter.java 97.36% <ø> (-0.14%) :arrow_down:
...m/github/housepower/buffer/SocketBuffedWriter.java 100.00% <ø> (ø)
...com/github/housepower/data/ColumnWriterBuffer.java 100.00% <100.00%> (ø)
.../com/github/housepower/serde/BinarySerializer.java 94.73% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 2234592...14eff7a. Read the comment docs.