SOSP'2015 Poster Submission

[Hoverbear]

@ycoady suggested we consider submitting a poster to SOSP'2015!

Location: Monterey, California, USA Submission deadline (HARD, NO EXTENSIONS): July 31, 2015 (23:59PM PDT) Acceptance notification: August 7, 2015 Conference: October 4-7, 2015

Details: http://www.ssrc.ucsc.edu/sosp15/callForPosters.html

Abstract

A Rust-Hulled Raft

Andrew Hobden (University of Victoria, undergraduate student, will present) Dan Burkert, James McGlashan, Yvonne Coady (University of Victoria)

Demonstration: Possible and prepared, executes on GEE, but not necessary.

Though the design of Ongaro and Osterhout’s Raft algorithm dramatically improves understandability of consensus relative to Paxos, an implementation can still be relatively challenging to validate and verify. Part of the problem stems from infrastructure bugs Engler identified in low-level systems. By leveraging higher-level language mechanisms, such as those in Go, Java, C++, C#, Scala, and Erlang, many implementations are bypassing these bugs, remaining true to the original Raft design, but ultimately compromising on performance or understandability. Our work evaluates the potential of Rust to build a fast, low-level implementation that is less susceptible to several classifications of Engler’s bugs and corresponds directly to the original design.

Specifically, we evaluate a prototype Raft library that interfaces with custom or preexisting Log and State Machine implementations, providing a Client, Server, and Consensus Module within its core. We focus on an asynchronous single threaded event loop model in the Rust language, allowing our implementation to have strong safety and performance characteristics with low demands. Communication, a primary performance concern, is kept as lightweight and efficient through Renshaw's Cap'n Proto implementation.

Our implementation features over 85% code coverage, and initial benchmarking on the Consensus module (no network traffic) shows that it scales well. Specifically, a proposal to a single node cluster takes approximately 1 microsecond, and a 7 node cluster approximately 115 microseconds. Additionally, we show how lost cost language features in Rust prevents many of the chronic bugs typically found in C based systems. We are currently exploring opportunities to further enhance trust and security, testing our implementation for failure conditions, and evaluating key features of the implementation relative to a number of other contemporary implementations of Raft in various high-level languages. (Note to reviewers: Our Rust-based Raft consensus has been a community based collaboration, and is currently scheduled as the topic for a Rust Meetup at Mozila Headquarters in late August. We will be including the feedback from this community in our poster if it is accepted, as well as expanding and improving benchmarking measurements).

Poster

[ycoady]

love it... making a mess with a few questions and edits, feel free to ignore!

Today our systems operate in extreme conditions, functioning across containers, virtual machines, infrastructure, networks, embedded systems, in our pockets, and even inside of us. Many of these systems depend on one another for operation, while others are able to operate in failing connectivity without disaster. In many cases it is preferable to have the latter, especially if the continued operation of the system is at stake. Distributed consensus represents one small part of a larger system, and offers the ability to maintain a replicated persistent log containing actions that are applied globally into a state machine. This allows n clients to communicate to a cluster of m servers in a stable and predictable manner, even in failing network conditions. Could be worth mentioning something about how the complexity of Paxos was uninviting, but now many more researchers are diving in with many more languages... or something?!

Using Ongaro and Osterhouts's Raft algorithm we are developing a fast, low level, low demands implementation of the system in an opinionated, minimal way. Might be worth clarifying exactly what you mean by "low demands" and "opinionated"? The Raft library interfaces with custom or preexisting Log and State Machine implementations, providing a Client, Server, and Consensus Module within its core. We have chosen an asynchronous single threaded event loop model in the Rust language, allowing our implementation to have strong safety and performance characteristics with low demands. Communication, a primary performance concern, is kept as lightweight and fast as possible by using Renshaw's Cap'n Proto implementation. TODO: Talk about performance tests and testing. Absolutely---if it is possible to show yours relative to any of the others on the Raft page, that would ROCK! Updating your row there before submitting would be great as well! We are currently exploring opportunities in trust and security as well as testing our implementation for further failure conditions.

b e a u t i f u l !

[dwrensha]

s/Rensha/David Renshaw/

[Hoverbear]

@dwrensha Gosh darnit I do that all the time, I'm so sorry!

From your twitter:

@dwrensha

The 'w' comes earlier than you'd think!

[Hoverbear]

@ycoady Once we get more features working and have more battle testing doing some performance measurements would be very useful, yeah. I remember Min was working on a consensus tester!

Moreso, though, I'm interested in testing reliablility and stablility.

[Hoverbear]

@james-darkfox want to be included in on this? No obligations from you other than keep contributing! :)

[WildCryptoFox]

@Hoverbear Sure!

[Hoverbear]

Updated abstract with @ycoady's feedback.

[Hoverbear]

Updated poster and abstract with some initial benchmarks. The abstract is too darn long!

[Hoverbear]

We submitted this! Closing for now. If we get accepted I'll open a new issue!

[Hoverbear]

The poster was rejected.

[WildCryptoFox]

Rejected due to any reason specified or undisclosed why?