attic-labs / noms

The versioned, forkable, syncable database
Apache License 2.0
7.45k stars 267 forks source link

Estimate List.copyReadAhead size using the compressed size #3740

Open kalman opened 6 years ago

kalman commented 6 years ago

List.copyReadAhead attempts to figure out the leaf chunk size in order to read 8MB chunks at a time: https://github.com/attic-labs/noms/blob/d1d73cd2915b86ebaffc735e0cf384d84c5e39da/go/types/list.go#L160 however this code isn't right because it uses the uncompressed chunk size, not the compressed chunk size.

Fixing this would require some plumbing.

@rafael-atticlabs