postmates / hopper

A Rust mpsc with unbounded members in bounded memory
Other
48 stars 7 forks source link

Use write bounds to denote non-continguous writes #5

Closed blt closed 7 years ago

blt commented 7 years ago

Previously hopper suffered fragmentation problems in its array of memories. In particular, it was possible for a receiver to lose writes and re-detect them at a later time in an out of order fashion. The basic access pattern of hopper has always assumed a continually growing "index" in memory, which was disk. With the introduction of the actual in-memory buffers it was possible for either the sender or the receiver to backtrack and then skip a chunk of memory to avoid reading in something from a later write.

The solution is to store the bounds of writes. A receiver keeps a rolling ID of its reads which the senders use to note when a new bound is needed, owing, potentially, to the current bound having been invalidated. So long as the sender captured receiver id remains the same the current write boundaries grow.

Resolves #4

Signed-off-by: Brian L. Troutwine blt@postmates.com

codecov-io commented 7 years ago

Current coverage is 93.54% (diff: 100%)

Merging #5 into master will increase coverage by 0.36%

@@             master         #5   diff @@
==========================================
  Files             4          4          
  Lines           352        372    +20   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits            328        348    +20   
  Misses           24         24          
  Partials          0          0          

Powered by Codecov. Last update 7c4fdfb...9d82960