search

cholcombe973 / rusix

Distributed filesystem in Rust
Other
54 stars 11 forks source link

Rework the API to use flatbuffers or CapNP depending on which is faster. #11

Open cholcombe973 opened 6 years ago

cholcombe973 commented 6 years ago

They both seem to satisfy the basic requirements of API versioning, backwards compatibility and fast throughput. With small IO operations it's critical that the API serialize and deserialize quickly so as not to introduce to much latency. Generally 4K writes in Ceph and Gluster are pretty terrible and there's a variety of reasons why but RPC overhead is a big part of it. The other big part is just physics of moving packets to replicas and waiting for the last writer to respond before responding to the client with success.

cgmcintyr commented 5 years ago

Which of these turned out to be better for use in a DFS?

cholcombe973 commented 5 years ago

I think flatbuffers ended up being the best in terms of serialization speed and network packet size.