RU Consumption is Unaffected when using Gateway Cache and Consistency Level Session

[robsierra]

Describe the bug Using Microsoft.Azure.Cosmos version 3.23.0-preview, when the CosmosClient is generated if it uses CosmosClientOptions.ConsistencyLevel = ConsistencyLevel.Session duplicate queries will not reduce the RU cost whatsoever, however it will reduce the RU cost to 0 if using ConsistencyLevel.Eventual.

To Reproduce Create a CosmosClient pointing to a CosmosDB resource with gateway caching enabled with the following options:

new CosmosClient({yourUri},
    {yourAuthKey},
    new CosmosClientOptions
    {
        ConnectionMode = ConnectionMode.Gateway,
        ConsistencyLevel = ConsistencyLevel.Session
    });

Then write a query against any database and container. Run the query more than once and check the FeedResponse.RequestCharge every time. Every time the query is executed RUs are charged against the account. Now create a ComosClient with the following options:

new CosmosClient({yourUri},
    {yourAuthKey},
    new CosmosClientOptions
    {
        ConnectionMode = ConnectionMode.Gateway,
        ConsistencyLevel = ConsistencyLevel.Eventual
    });

Repeat the steps above and you'll see that after the first request the RU charge drops down to 0 for subsequent requests.

Expected behavior According to this documentation: https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-configure-integrated-cache#adjust-request-consistency

The integrated cache should be utilized if the ConsistencyLevel is Eventual or Session

Actual behavior The integrated Cache appears to be being bypassed when the ConsistencyLevel is Session

Environment summary SDK Version: 3.23.0-preview OS Version Windows

Additional context Additionally, you may want to consider not exposing CosmosClient.ClientOptions.ConnectionMode and CosmosClient.ClientOptions.ConsistencyLevel because if those variables are not set in the constructor for CosmosClient they seem to be ignored and that can be quite confusing.

[ealsur]

SDK is not related to behavior of integrated cache

[ealsur]

@timsander1 Do you know who can help regarding Integrated Cache?

[robsierra]

@ealsur This is a confusing concept as the functionality clearly changes based on how the variables are set in the SDK, certainly they're at least related.

[ealsur]

What I mean is that the SDK has no inference in the Integrated Cache and any RU saving, that is complete logic applied on the Integrated Cache layer.

When you change the consistency on an operation, all the SDK does is send that information to the backend so the consistency is enforced, again, there is no logic in the SDK that it is applying.

The SDK in this scenario is just the medium through which your configuration is being sent to the backend that has Integrated Cache enabled, the RU consumption is part of the service response, not calculated nor altered on the SDK. If using Eventual yields 0 RU but using Session yields X RU, that sounds like Integrated Cache only caches Eventual requests.

[robsierra]

@ealsur I see, that does make sense and thank you for the explanation. If that is the case I would appreciate any help directing me to whom I should make a bug report.

[timsander1]

Hi @robsierra, thanks for reporting this! I've flagged this to the Cosmos DB integrated cache engineering team, who will investigate. I'll update here in the next few days.

[robsierra]

@timsander1 Thank you very much!

[timsander1]

Hi @robsierra, a couple following up questions:

1) are most of your reads within the same client? Or different clients? 2) Which SDK version are you using? .NET SDK 3.23 preview, correct?

[robsierra]

@timsander1

1. The same client, we were testing this specific functionality so we only had one instance of the client loaded.
2. Yes, we made sure we were using the most up to date sdk which was 3.23 preview.

[timsander1]

Thanks! Could you also share:

1. Sample code and/or repro steps (if it doesn't easily repro)
2. Your Cosmos DB account name

You're welcome to share this here but if you'd prefer you can email me at tisande at microsoft dot com

[robsierra]

I will email you our CosmosDB name but I'm including the sample code here for reference in the event others experience this issue.

async Task Main()
{
    var client = new CosmosClient("{connectionUrl}",
        "{authKey}",
        new CosmosClientOptions
        {
            ConnectionMode = ConnectionMode.Gateway,
            ConsistencyLevel = ConsistencyLevel.Session
        });

    Database database = client.GetDatabase("{dbName}");
    Container container = database.GetContainer("{containerName}");

    double totalRU = 0;
    var cmd = "select * from T where T.Timestamp > '2021-11-10'";
    var query = new QueryDefinition(cmd);
    using (FeedIterator<dynamic> feedIterator = container.GetItemQueryIterator<dynamic>(query,
    requestOptions: new QueryRequestOptions
    {
        // Only available using preview sdk
        DedicatedGatewayRequestOptions = new DedicatedGatewayRequestOptions
        {
            MaxIntegratedCacheStaleness = TimeSpan.FromMinutes(30),

        }
    }))
    {
        while (feedIterator.HasMoreResults)
        {
            FeedResponse<dynamic> response = await feedIterator.ReadNextAsync();
            totalRU += response.RequestCharge;
        }
    }
    totalRU.Dump();
}

This is run in Linqpad, if you run the above code twice you will be charged RU's both times, but if you set the consistencyLevel to eventual and run the code twice you will only be charged for the first request.

[seesharprun]

I'm experiencing the same behavior with both ItemRequestOptions and QueryRequestOptions. I was able to reproduce multiple times by creating a new Cosmos DB SQL API account, and using the following code:

using Microsoft.Azure.Cosmos;

string connectionString = "<gateway-connection-string>";

CosmosClientOptions options = new()
{
    ConnectionMode = ConnectionMode.Gateway
};

CosmosClient client = new (connectionString, options);

Container container = client.GetContainer("cosmicworks", "products");

ItemRequestOptions operationOptions = new()
{
    ConsistencyLevel = ConsistencyLevel.Session
};

string id = "<guid>";
PartitionKey partitionKey = new ("<partition-key>");

ItemResponse<Product> response = await container.ReadItemAsync<Product>(id, partitionKey, requestOptions: operationOptions);

Console.WriteLine($"Request charge:\t{response.RequestCharge:0.00} RU/s");

If I set the consistency level at the account level to a default of session, I'm getting the same behavior.

I'm only able to verify the cache is working by changing the consistency-level on a per-request basis:

ConsistencyLevel = ConsistencyLevel.Eventual

Then, I will see 0 RUs on the second request. I'm not running anything mission critical, it just seemed to run counter to what the documentation suggested.

[jinsaini]

@timsander1 Sorry if I have misunderstood the usage of caching but what happens if a query is cached but then an item is added to the data that should return in the query. EG

1) select * from orders where customer=12345 -> Gets added to cache

2) select * from orders where customer=12345 -> Returns cached data - this is fine

3) insert new order for customer 12345

4) select * from orders where customer=12345 -> returns cached results but does not include the newly added item

is there any way that the insert can invalidate the cache so (4) re-populates the cache?

Many thanks