search

wangkuiyi / recordio

Apache License 2.0
11 stars 2 forks source link

WIP Remove a chunk buffer in parseChunk #54

Closed wangkuiyi closed 5 years ago

wangkuiyi commented 5 years ago

This PR saves a buffer that holds the compressed chunk data with the cost of making the reading slightly slower.

Before this PR:

yi@WangYis-iMac:/go/src/github.com/wangkuiyi/recordio (save_space_read)$ go test -bench=.
goos: darwin
goarch: amd64
pkg: github.com/wangkuiyi/recordio
BenchmarkRead/reading_records_00000_to_00050-4                 5     295303520 ns/op
BenchmarkRead/reading_records_00050_to_00100-4                 3     446877315 ns/op
BenchmarkRead/reading_records_00100_to_00150-4                 5     294625298 ns/op
PASS
ok      github.com/wangkuiyi/recordio   8.315s

After this PR:

yi@WangYis-iMac:/go/src/github.com/wangkuiyi/recordio (save_space_read)$ cp /tmp/chunk.go .
yi@WangYis-iMac:/go/src/github.com/wangkuiyi/recordio (save_space_read)*$ go test -bench=.
goos: darwin
goarch: amd64
pkg: github.com/wangkuiyi/recordio
BenchmarkRead/reading_records_00000_to_00050-4                 5     311463509 ns/op
BenchmarkRead/reading_records_00050_to_00100-4                 3     472621864 ns/op
BenchmarkRead/reading_records_00100_to_00150-4                 5     315371342 ns/op

Given the speed slow down is slight, I prefer to merge this PR to save space. What do you think, reviewer?

wangkuiyi commented 5 years ago

Use https://github.com/wangkuiyi/recordio/pull/57 instead.