Closed sadikovi closed 6 years ago
@sunchao I implemented page writer. Let me know if I need to add more tests, I feel like the currently added tests may not cover all of the cases. Thanks!
Files with Coverage Reduction | New Missed Lines | % | ||
---|---|---|---|---|
encodings/encoding.rs | 1 | 94.8% | ||
column/page.rs | 4 | 96.3% | ||
<!-- | Total: | 5 | --> |
Totals | |
---|---|
Change from base Build 558: | 0.05% |
Covered Lines: | 11348 |
Relevant Lines: | 11890 |
@sunchao can you have a look again? I addressed your comments and added more docs describing each method.
I was thinking if PageWriter
trait needs an improvement; it might be a bit difficult to reason about the API without column writer. Let me know what you think.
Thanks!
Yes, you are right, it does. We should be able to refactor both when working on column chunk writer.
Merged. Thanks @sadikovi !
Thanks @sunchao!
This PR adds:
PageWriter
interface and serialised implementation.CompressedPage
in addition toPage
. Compressed page is a wrapper that allows us to store compressed buffer + uncompressed length. Page, when created, is always assumed to have uncompressed buffer. This is not the case for compressed page - internally we store compressed data.SerializedPageReader
to takeT: Read
instead of file source. This allows me to test a page roundtrip.