apple / foundationdb

FoundationDB - the open source, distributed, transactional key-value store
https://apple.github.io/foundationdb/
Apache License 2.0
14.18k stars 1.29k forks source link

Automatic idempotency ids have insufficient entropy #11446

Open atn34 opened 3 weeks ago

atn34 commented 3 weeks ago

The automatically generated ids are 128-bit, but the PRNG's seed is only 32-bits. If clients only commit one transaction per process/network thread, then each transaction's id should be fully determined by the 32-bit seed. According to the birthday problem we only need about 82137 transactions in the unexpired window of idempotency ids until we expect to see a collision.

I haven't tried to reproduce this yet, but I expect an effective mitigation to be setting the hidden idempotency_id option with sufficient entropy prior to setting the automatic_idempotency option.

There may be other places we can run into UID collisions because of 32-bit seeds. I don't know if using 128-bit seeds is sufficient to mitigate all this, but it would definitely be better.