jkosh44
commented
3 years ago If anyone has a particular format that I left out that they think would be good, please let me know.
Open jkosh44 opened 3 years ago
jkosh44
commented
3 years ago If anyone has a particular format that I left out that they think would be good, please let me know.
jkosh44
commented
3 years ago branch: https://github.com/jkosh44/noisepage/tree/bson
UPDATE: The original JSON implementation was converting the record contents itself to and from CBOR. The original numbers used for BSON kept that conversion in. The updated numbers remove that conversion.
| Replication | Durability | Modifications | Log Throughput (records per millisecond) |
|---|---|---|---|
| Sync | Sync | None | 3.8700566481216665 |
| Async | Sync | None | 30.736948475102366 |
| Async | Async | None | 29.028352465595663 |
| Durability | Log Throughput (records per millisecond) |
|---|---|
| Sync | 3.8883037697513125 |
| Async | 3.874525911 |
jkosh44
commented
3 years ago branch: https://github.com/jkosh44/noisepage/tree/messagepack
| Replication | Durability | Modifications | Log Throughput (records per millisecond) |
|---|---|---|---|
| Sync | Sync | None | 3.8606188627282036 |
| Async | Sync | None | 29.38279984543937 |
| Async | Async | None | 28.471362156671457 |
| Durability | Log Throughput (records per millisecond) |
|---|---|
| Sync | 3.887471136 |
| Async | 3.91150189 |
jkosh44
commented
3 years ago branch: https://github.com/jkosh44/noisepage/tree/ubjson
| Replication | Durability | Modifications | Log Throughput (records per millisecond) |
|---|---|---|---|
| Sync | Sync | None | 3.864162520678762 |
| Async | Sync | None | 30.148788480999627 |
| Async | Async | None | 28.387447086724443 |
| Durability | Log Throughput (records per millisecond) |
|---|---|
| Sync | 3.887471136 |
| Async | 3.91150189 |
jkosh44
commented
3 years ago branch: https://github.com/jkosh44/noisepage/tree/cbor
| Replication | Durability | Modifications | Log Throughput (records per millisecond) |
|---|---|---|---|
| Sync | Sync | None | 3.861907674897514 |
| Async | Sync | None | 30.714806087560053 |
| Async | Async | None | 28.913739773546347 |
| Durability | Log Throughput (records per millisecond) |
|---|---|
| Sync | 3.918585055891593 |
| Async | 3.9006568057238713 |
Summary
Currently, we serialize all messages related to replication using JSON. The implementation can be found here: https://github.com/cmu-db/noisepage/blob/97eb7ecc83785ed57cc02a14e3d63b553b252e2e/src/include/replication/replication_messages.h#L36-L108 https://github.com/cmu-db/noisepage/blob/97eb7ecc83785ed57cc02a14e3d63b553b252e2e/src/replication/replication_messages.cpp#L19-L56
Turning replication on causes a significant slowdown to the database and one of the primary causes is the JSON serialization of messages. Below are some performance results of running TPCC on dev10 with 8 threads with various database configurations:
Below are some metrics on log throughput for the primary node with various database configurations:
Just for reference below are some metrics on log throughput for the replica node
Solution
A solution to this is to switch to a different message format than JSON and I plan on investigating a handful of alternatives and their impact on log throughput and request throughput.
Nlohmann
We use the Nlohmann JSON package to implement JSON in NoisePage. This package comes with a bunch of other binary formats built into the package. It's probably worth trying all of these since they can each be implemented with a couple of changed lines. Some require you to first convert your data to JSON before converting to a different binary format, and it's unclear to me if this has a significant performance penalty compared to converting directly to the message format.
Alternatives
Below are a handful of message formats I have found from some brief research. I plan on narrowing this down to roughly 4 after some more research.
Dependency Bloat
One of the considerations when implementing a new message format will be dependency bloat. I don't plan on coming up with my own implementation for any of these formats so we'll have to bring in third-party libraries. It will be important to make sure we don't bring in more than necessary to avoid dependency bloat.