facebook / akd

An implementation of an auditable key directory
Apache License 2.0
243 stars 36 forks source link

Adding lookup / history proof efficiency optimization #444

Closed kevinlewi closed 3 months ago

kevinlewi commented 3 months ago

Implements an efficiency optimization that changes the contents of a key history proof (reducing the number of total past marker versions and future marker versions) without compromising on security.

From the doc comments of get_marker_versions() (which is basically the only function that is updated):

The past marker versions are determined as follows:

  1. Include the largest power of 2 that is less than start_version.
  2. Include the largest element of MARKER_VERSION_SKIPLIST that is less than start_version.
  3. Include at most a log_2(start_version) number of versions between start_version and the largest power of 2 less than start_version, determined as follows: For each bit position i in start_version, if the bit is 1, include the value of start_version with the ith bit set to 0 and followed by trailing zeros.

As a concrete example, if start_version = 85, the past marker versions would be [16, 64, 80, 84]. Since: 01010101 => 85 01010100 => 84 01010000 => 80 01000000 => 64

And 16 comes from MARKER_VERSION_SKIPLIST.

The future marker versions are determined as follows:

  1. Include all powers of 2 that begin from start_version, up until the smallest element in MARKER_VERSION_SKIPLIST that is greater than start_version.
  2. Include all elements of MARKER_VERSION_SKIPLIST that are between start_version and epoch.
  3. Include at most a log_2(start_version) number of versions between start_version and the smallest power of 2 greater than start_version, determined as follows: For each bit position i in start_version, if the bit is 0, include the value of start_version with the ith bit set to 1 and followed by trailing zeros.

As a concrete example, if start_version = 85, the future marker versions would be [86, 88, 96, 128, 256, 65536, 2^32] (potentially truncated depending on if any of these numbers exceed epoch).

Since: 01010101 => 85 01010110 => 86 01011000 => 88 01100000 => 96 10000000 => 128

And the remainder of the list comes from MARKER_VERSION_SKIPLIST.

Note that the past marker versions do not contain start_version, as this would be redundant in the history proof (since membership is already checked for start_version).

codecov-commenter commented 3 months ago

Codecov Report

Attention: Patch coverage is 97.79412% with 3 lines in your changes missing coverage. Please review.

Project coverage is 88.04%. Comparing base (3ce5335) to head (a73b230). Report is 18 commits behind head on main.

Files with missing lines Patch % Lines
akd_core/src/utils.rs 97.79% 3 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #444 +/- ## ========================================== - Coverage 88.61% 88.04% -0.57% ========================================== Files 39 38 -1 Lines 9109 8282 -827 ========================================== - Hits 8072 7292 -780 + Misses 1037 990 -47 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

kevinlewi commented 3 months ago

@dillonrg thanks for the feedback!