Session-level consistency across multiple SDK clients/across different processes

[d6teo]

Describe the bug We have a Single-region account with Session consistency that is utilized by multiple v3 SDK clients (across multiple processes). All SDK clients live inside APIs hosted in Azure K8s.

After executing a large number of write operations using an SDK client (client X), some of the other SDK clients (clients Y,Z,...) do not fetch the items included in some of the write operations of client (X).

• A stored procedure is used to generate & execute the query
• Have tried querying with clients (Y,Z,...) predating client (X), as well as clients (Y,Z,...) instantiated after client (X). Some clients (e.g. only client Y) will keep on omitting the items written by client (X)
• Querying from Cosmos portal includes items written by client (X) in the result set
• CosmosClient is setup with the configuration below:

  CosmosClientOptions connectionPolicy = new() 
  {
  ConnectionMode = ConnectionMode.Direct,
  };
  return new CosmosClient(<endpoint>, <key>, connectionPolicy);

This is more of a question rather than a bug report: -Is there any guarantee/threshold/estimation of how long it may take for different sessions to "converge"(produce the same result set for the same query)? -Is there any way to update session info on SDK clients apart from using session tokens (e.g. recycling an SDK client)?

Expected behavior Clients Y,Z,... should produce consistent result sets for the same query after a certain amount of time.

Actual behavior Clients Y,Z,... produce inconsistent result sets for the same query several hours after the write operations.

Environment summary Azure CosmosDB account region: South Central US SDK Version: 3.29.0 SDK host: Docker container/Azure K8S OS Version (e.g. Windows, Linux, MacOSX): Debian GNU/Linux 11 (bullseye)

[ealsur]

Based on the public documentation: https://learn.microsoft.com/en-us/azure/cosmos-db/consistency-levels#session-consistency

... This guarantee assumes a single “writer” session or sharing the session token ...

If the client didn't initiate a write to a physical partition, the client doesn't contain a session token in its cache and reads to that physical partition behave as reads with Eventual Consistency...

What you are describing is that the writer of the initial operation is not passing the SessionToken to the other clients, thus, you are really using Eventual Consistency.

There are no guarantees in that consistency of when there will be convergence.

[d6teo]

@ealsur Thanks for the explanation & for providing the link to documentation.