Open auniverseaway opened 8 months ago
What are the limitations in the size and number of KV entries allowed in a given "bucket"?
What are the limitations in the size and number of KV entries allowed in a given "bucket"?
None. It's infinite. See: https://developers.cloudflare.com/kv/platform/limits/
Somewhat related: we have a single KV for all orgs due to the Worker / KV binding in the wrangler.toml
. I think part of this exploration should be:
Intro
This will be a bit of solutioning, so apologies in advance. This isn't meant to be prescriptive, it's only meant to illustrate how I think we could solve this. All other ideas are welcome.
Vocabulary
Authentication - We know who the person is. Authorization - We know what the person can do.
Background
As of today, we use Cloudflare KV for authentication (technically a very light authorization) storage. This currently only exists at the org / bucket level. The user can either do everything in the bucket, or they cannot see / do anything. Part of the reason we used KV is that it seems to be more performant than finding an S3 JSON file to determine permissions.
And idea
I think this would need to be validated, but I almost wonder if we can expand the KV use for all descendants.
The idea would basically be:
/adobecom/bacom/de/products/experience-manager/aem-assets
.Assumptions
From what I have seen in the wild on many projects, custom permissions do not typically go lower than 4-5 levels deep. We could even ignore org for certain requests that are below a site.
In this scenario, we would descend until we don't find a KV that matches the sub-path. In theory, this would be more performant than trying to walk up the tree because a lot of the leafs will be null.
Other ideas
DynamoDB? S3 and see the performance implications?
Other notes
It would be good to get concrete about how fast KV is in comparison to the alternatives. My general hope is that we stay around ~300ms to deliver the response. In my, admittedly brief, explorations, I saw two S3 calls ballooning up to 500ms+.
\cc @bstopp @cazzaranjosh