danthegoodman1 / httpkv

A really simple KV that runs off HTTP
0 stars 0 forks source link

Use FDB Versionstamp for CAS #3

Open danthegoodman1 opened 4 months ago

danthegoodman1 commented 4 months ago

Could use a Versionstamp suffix for the key, then a get will just get the latest version of the key?

Not sure how to get latest key, maybe range where you get from reverse, limit 1, and the end is the key + \xff

danthegoodman1 commented 4 months ago

Likely not needed as this adds complexity to parsing the key (both for reads and v requests), as well as some FDB-specific stuff.

In theory we could append the versionstamp to the body too, then rip it off for reads.

If we can trust the timestamp of the server as "close enough" then it should be fine, but need to be careful of drift (not a likely issue with main cloud providers). If we need certainty the best method is probably to append the versionstamp to the body using https://pkg.go.dev/github.com/apple/foundationdb/bindings/go/src/fdb#Transaction.SetVersionstampedValue

danthegoodman1 commented 4 months ago

For versionstamped value either set it to the first 10(14) bytes as the prefix of the value or calculate the length of the real value and append the right bytes to put it at the end (will need to pop 10 bytes when reading)

SetVersionstampedValue:

Transforms “param“ using a versionstamp for the transaction. Sets the “key“ given to the transformed “param“. The parameter is transformed by removing the final four bytes from “param“ and reading those as a little-Endian 32-bit integer to get a position “pos“. The 10 bytes of the parameter from “pos“ to “pos + 10“ are replaced with the versionstamp of the transaction used. The first byte of the parameter is position 0. A versionstamp is a 10 byte, unique, monotonically (but not sequentially) increasing value for each committed transaction. The first 8 bytes are the committed version of the database (serialized in big-Endian order). The last 2 bytes are monotonic in the serialization order for transactions. WARNING: At this time, versionstamps are compatible with the Tuple layer only in the Java, Python, and Go bindings. Also, note that prior to API version 520, the versionstamp was always placed at the beginning of the parameter rather than computing an offset.