BBVA / qed

The scalable, auditable and high-performance tamper-evident log project
https://qed.readthedocs.io/
Apache License 2.0
94 stars 19 forks source link

Move event hashing out of balloon internals #136

Closed jllucas closed 5 years ago

jllucas commented 5 years ago

Move hashing from balloon internals (balloon.go) to Raft (raft.go), just before inserting the command to the WriteAheadLog.

Others:

jllucas commented 5 years ago

BenchmarkRaftAdd with b.N = 2000000

HASHING OUTSIDE BALLOON

event := utilrand.Bytes(128)

| => du -sh /var/tmp/raft-test/node1/raft/wal/*

176M /var/tmp/raft-test/node1/raft/wal/000003.log 52K /var/tmp/raft-test/node1/raft/wal/LOG

event := utilrand.Bytes(2048)

| => du -sh /var/tmp/raft-test/node1/raft/wal/*

176M /var/tmp/raft-test/node1/raft/wal/000003.log 52K /var/tmp/raft-test/node1/raft/wal/LOG

HASHING INSIDE BALLOON (MASTER BRANCH)

event := utilrand.Bytes(128)

| => du -sh /var/tmp/raft-test/node1/raft/wal/*

224M /var/tmp/raft-test/node1/raft/wal/000003.log 145M /var/tmp/raft-test/node1/raft/wal/000011.log 52K /var/tmp/raft-test/node1/raft/wal/LOG

event := utilrand.Bytes(2048)

| => du -sh /var/tmp/raft-test/node1/raft/wal/*

256M /var/tmp/raft-test/node1/raft/wal/000003.log 257M /var/tmp/raft-test/node1/raft/wal/000011.log 257M /var/tmp/raft-test/node1/raft/wal/000012.log 257M /var/tmp/raft-test/node1/raft/wal/000014.log 257M /var/tmp/raft-test/node1/raft/wal/000015.log 257M /var/tmp/raft-test/node1/raft/wal/000017.log 257M /var/tmp/raft-test/node1/raft/wal/000018.log 256M /var/tmp/raft-test/node1/raft/wal/000020.log 256M /var/tmp/raft-test/node1/raft/wal/000021.log 256M /var/tmp/raft-test/node1/raft/wal/000023.log 257M /var/tmp/raft-test/node1/raft/wal/000024.log 257M /var/tmp/raft-test/node1/raft/wal/000026.log 256M /var/tmp/raft-test/node1/raft/wal/000027.log 257M /var/tmp/raft-test/node1/raft/wal/000029.log 257M /var/tmp/raft-test/node1/raft/wal/000030.log 256M /var/tmp/raft-test/node1/raft/wal/000032.log 97M /var/tmp/raft-test/node1/raft/wal/000033.log 496M /var/tmp/raft-test/node1/raft/wal/000034.sst 113M /var/tmp/raft-test/node1/raft/wal/000035.sst 132K /var/tmp/raft-test/node1/raft/wal/LOG