search

ning / compress

High-performance, streaming/chunking Java LZF codec, compatible with standard C LZF package
Other
254 stars 40 forks source link

Implement skip() efficiently, without needing to decode if possible #19

Closed cowtowncoder closed 12 years ago

cowtowncoder commented 12 years ago

Currently InputStream.skip(...) functionality uses same code path as reads, which works functionally, but does unnecessary work of decoding. For some use cases it would be really great to do "coarse skip", since format allows one to completely skip a chunk without decoding, as the length is known on per-chunk basis. Depending on underlying storage format, this could possibly allow skipping some reads too.

Once implemented, it would be also be good to see if a file-based approach could do even better and use random-access for access. But first steps first.

cowtowncoder commented 12 years ago

Implemented -- nice 6x improvement for skipping performance on tests...