Open gf2121 opened 12 months ago
Note that oal.store.ByteBuffersDataOutput
takes a different and neat approach to gracefully growing: it picks an initial block size (1024 bytes by default), and appends new blocks as you write bytes, but then if it reaches 100 blocks, it "resizes" itself by doubling the block size and copying over, so that now you have 50 blocks.
So it's still O(N)
amortized cost of that doubling/copying with time, and at any given moment you will not be wasting too many %tg of the bytes you've written, except at the start 1 KB block size.
In https://github.com/apache/lucene/pull/12624, I moved the main FST body out of BytesStore
into ByteBuffersDataOutput
, and BytesStore becomes only a single byte[]
for the currently written node so maybe we don't need to do this?
Description
From https://github.com/apache/lucene/pull/12604#discussion_r1344639608