mdrichardson
commented
4 years ago I've got some thoughts specific to your points at the bottom. However, Cosmos partitioning is commonly misunderstood, so here's a quick primer to get everybody on the same page:
Key Term Clarification
For Bot Framework purposes, “container” is basically synonymous with “collection”. A container can store a single collection, graph, or table. Bot Framework uses a Cosmos container that stores a collection.
“Document” refers to an item stored in Cosmos as a whole, and not the “document” property of a state object.
Partition Keys will be referred to as PK and Partition Values as PV, to more clearly distinguish from other “keys” and “values”. If we have a document:
{
"id": 1,
"data": "xyz",
"myKey": "myValue"
} “myKey” is the PK and “myValue” is the PV. In Cosmos Data Explorer, it would look like:
In the right-hand data explorer document viewer:
1 { 2 "id": 1. 3 "data": "xyz", 4 "myKey": "myValue", 5 }
and in the left-hand side document selector
| id | /myKey |
|---|---|
| 1 | myValue |
Cosmos Partitioning with Partition Keys
Here’s a basic layout of a Cosmos DB:
A database can have multiple containers, but generally, Bot Framework users will only use one container per bot.
Each container has multiple physical partitions. The benefit of multiple physical partitions is that the data is split up so that when querying, you have a smaller subset of data to look over and don’t have to search the entire container. CosmosDB dynamically adjusts the number of physical partitions per container based on throughput and storage needs. The user has absolutely no control over this, and it all happens on the backend.
Logical Partitions consist of a set of documents with the same PVs and there is a Logical Partition for every unique PV. Since every document that has a PV of “myValue” will always reside within the same Logical Partition and also because Logical Partitions cannot be distributed across multiple Physical Partitions, every document with the same PV will reside on the same Physical Partition.
Cosmos DB automatically distributes data across Physical Partitions based on PVs, so setting the right PK is important to ensure that documents get distributed such that each Physical Partition uses roughly the same throughput and storage.
In addition to making sure a Physical Partition doesn’t become a hotspot due to read/writes/storage, the right PK ensures that queries happen across Physical Partitions as rarely as possible. It’s important to think of PKs as providing load balancing benefits and cross-partition avoidance, whereas partitioning, in general, is what provides indexing and throughput benefits.
Note: The number of Logical Partitions (which are created by unique PVs) does not affect performance. Only the balance across Physical Partitions does.
Default Partition Keys and the Bot Framework
You cannot write more than one document at a time. Currently, the Bot Framework doesn’t query more than one document at a time. Therefore, any single storage operation that the Bot Framework makes is not cross-partition (with the current Bot Framework Storage implementation).
So, the only PK that really matters to the Bot Framework, currently, is one that distributes the documents evenly—we don’t have to worry about cross-partition queries because we don’t query more than one document at a time.
The default PK of "id" currently makes the most sense since we always know it and they’re basically 1 “id” : 1 Document, so they will distribute very evenly.
I know that conceptually, it doesn’t seem to make sense. But again, we only query 1 document at a time, so there is no reason to try to lump documents together under the same PV. Trying to use anything else just introduces additional code (“id” already exists on every document and doesn’t need to be modified) or lumps documents into Logical Partitions that may create hotspots.
References
- Official Docs – Partitioning Overview
- Official Docs – Partitioning and Horizontal Scaling
- Official Docs – Creating a Synthetic Key
- Official Blog – Azure Cosmos DB Partitioning Design Patterns
- Official Blog – Design Patterns – IoT and Aggregation
- 3rd-Party Blog – Data Partitioning Strategy in Cosmos DB
Rethink partition strategy - currently there is one document per partition. This seems to be inefficient on the one side regarding the usage of partitions but also on the consumption side, as partitions are a great way to optimize CosmosDB usage (or not).
As explained above, since we only query one document at a time, one document per partition is optimal for current Bot Framework State Storage implementations.
I could possibly see some benefit to partitioning by user and when any query is made for the user, we pull all of their conversations (or most recent) in the same query. I'm not sure how that would work on the Bot State side...maybe instead of calling UserState.SaveChanges() and ConversationState.SaveChanges(), we allow a BotState.SaveChanges() that acts on both at the same time or something.
Rethink naming of documents - the names of the documents are not predictable and therefore point reads are not possible. Being able to use point reads would greatly improve costs and performance.
Add an explicit property that describes the type of the document (e.g. "user" or "conversation"). Currently it is necessary to extract that from the id what is highly inefficient.
Agreed on both. This would have backwards-compat issues that would need to be worked around.
dr-dolittle
sgellock
cwhitten
carlosscastro
compulim
When using CosmosDB as storage layer different types of documents are being created per bot (e.g. users or conversations). Would it be possible to optimize the way they are stored for more efficient postprocessing?
Here are some suggestions:
Rethink partition strategy - currently there is on document per partition. This seems to be inefficient on the one side regarding the usage of partitions but also on the consumption side, as partitions are a great way to optimize CosmosDB usage (or not).
Rethink naming of documents - the names of the documents are not predictable and therefore point reads are not possible. Being able to use point reads would greatly improve costs and performance.
Add an explicit property that describes the type of the document (e.g. "user" or "conversation"). Currently it is necessary to extract that from the id what is highly inefficient.
Thanks
@goergenj @benbrown @sgellock