webcomponents / webcomponents.org

Home of the web components community
https://www.webcomponents.org
Apache License 2.0
359 stars 95 forks source link

[catalog-server] Ensure DB partitioning for tests and staging purposes #1351

Open justinfagnani opened 1 year ago

justinfagnani commented 1 year ago

We want to be able to partition the Firestore database for running multiple tests agains the same database in parallel, and to potentially support multi-tenancy, different environments in the same GCP project, etc.

To do this we need to ensure that each collection has a unique name per environment. We do this now with a namespace suffix on the packages collection, but this isn't quite good enough because collection group queries could be performed on other collections. So we need to make sure that .collection() and .collectionGroup() are never called directly, but called through utility methods that add the namespace suffix.

We also need to update the design doc to describe this pattern.

justinfagnani commented 1 year ago

There's a problem with expressing namespace as collection suffixes: indices. Indices must be created via the console or CLI, and take a few minutes to setup even on empty databases. Indices are also configured by collection IDs, and are required for collection group queries.

The other approach we can take is to add a namespace field to every document and always filter on that field.

justinfagnani commented 1 year ago

And a problem with using a field is that we have a few places where we get a document by building a reference with known document IDs, and not by querying. We need collection suffixes to keep references unique.

So, we can do this:

justinfagnani commented 1 year ago

This wasn't fully fixed yet