abonander / buf_redux

A drop-in replacement for Rust's std::io::BufReader, with extra features
Apache License 2.0
38 stars 16 forks source link

0.7: Implement Ringbuffers Using `slice-deque`, consolidate Strategies into Policies #6

Closed abonander closed 6 years ago

abonander commented 6 years ago

This 0.7.0 release adds a major new feature, and some other significant changes:

Ringbuffers and slice-deque

At @gnzlbg's recommendation, I implemented support for ringbuffers using his slice-deque crate (exposed by a target-feature of the same name), which uses virtual-memory tricks to create a buffer that loops around on itself in memory, such that consuming data from the head simultaneously makes more room at the tail. .make_room() on buffered types subsequently becomes a no-op. However, this has some caveats:

Still, this may be beneficial with the new policies, like MinBuffered, that will read into the buffer and consume bytes from it without completely emptying it. Any buffered type can be made to utilize a ringbuffer simply by using the ::new_ringbuf() or ::with_capacity_ringbuf() constructors instead of ::new() or with_capacity(), respectively.

Strategies Consolidated into Policies

The ReadStrategy and MoveStrategy traits are now consolidated into the ReaderPolicy trait, and the FlushStrategy trait is renamed to WriterPolicy for consistency. The callbacks of these traits get a mutable reference to the internal Buffer so they can make room or reserve space if necessary. These traits and some provided implementations are in the new policy module.

StdPolicy is provided as the default implementation of both ReaderPolicy and WriterPolicy, which implements the std::io behaviors for BufReader and BufWriter, respectively.

MinBuffered is a new ReaderPolicy which ensures there is at least the given number of bytes in the buffer every time .fill_buf() is called until the source reader is out of data. This is useful for parsing applications which require a certain amount of lookahead.

The given WriterPolicy implementations are mostly unchanged in behavior from their FlushStrategy days, though some have been renamed.

On top of all this, test coverage and documentation has been significantly improved.

closes #5

gnzlbg commented 6 years ago

but if address space is at a premium in your application then this may be a concern.

I would add here that this concern is particularly relevant for 32-bit platforms where one "only" has 4 Gb of virtual memory.

abonander commented 6 years ago

Published as 0.7.0