OrleansContrib / Orleans.CosmosDB

Orleans providers for Azure Cosmos DB
MIT License
40 stars 30 forks source link

Reminders needs a partition key to support shared throughput #22

Open dealproc opened 5 years ago

dealproc commented 5 years ago

With the case of wanting to store both the Application Data and the Orleans data within the same CosmosDB instance, where that CosmosDB instance is using the new "shared" throughput facilities, it is a requirement to set partition keys for each collection, according to the output [below].

Lifecycle start canceled due to errors at stage 20000
warn: Orleans.Runtime.Scheduler *stg/22/00000016.WorkItemGroup[101215]
      Task [Id=11, Status=RanToCompletion] in WorkGroup [SystemTarget: S10.1.1.18:11111:282256074*stg/22/00000016@S00000016] took elapsed time 0:00:01.1214201 for execution, which is longer than 00:00:00.2000000. Running on thread System.Threading.Thread
dbug: Orleans.Runtime.DeploymentLoadPublisher[0]
      UpdateRuntimeStatistics from S10.1.1.18:11111:282256074
   at Orleans.LifecycleSubject.OnStart(CancellationToken ct)
   at Orleans.Runtime.Scheduler.AsyncClosureWorkItem.Execute()
   at Orleans.Runtime.Silo.StartAsync(CancellationToken cancellationToken)
   at .....Silo.Program.StartSilo() in .....Silo\Program.cs:line 154
   at .....Silo.Program.Main(String[] args) in .....Silo\Program.cs:line 130
--------------------------------------------------------------------------------
Provisioned throughput collection should have a partition key
ActivityId: 44fa21ea-8e0a-43ad-a6e2-a5bf0d909d80, Microsoft.Azure.Documents.Common/2.2.0.0, Linux/9 documentdb-netcore-sdk/1.9.1
   at Microsoft.Azure.Documents.Client.ClientExtensions.ParseResponseAsync(HttpResponseMessage responseMessage, JsonSerializerSettings serializerSettings)
   at Microsoft.Azure.Documents.GatewayStoreModel.<>c__DisplayClass20_0.<<InvokeAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.<>c__DisplayClass1_0.<<ExecuteAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetry(Func`1 callbackMethod, Func`3 callShouldRetry, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetry(Func`1 callbackMethod, Func`3 callShouldRetry, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteAsync(Func`1 callbackMethod, IRetryPolicy retryPolicy, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.GatewayStoreModel.InvokeAsync(DocumentServiceRequest request, ResourceType resourceType)
   at Microsoft.Azure.Documents.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request)
   at Microsoft.Azure.Documents.Client.DocumentClient.CreateAsync(DocumentServiceRequest request)
   at Microsoft.Azure.Documents.Client.DocumentClient.CreateDocumentCollectionPrivateAsync(String databaseLink, DocumentCollection documentCollection, RequestOptions options)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.<>c__DisplayClass1_0.<<ExecuteAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetry(Func`1 callbackMethod, Func`3 callShouldRetry, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteRetry(Func`1 callbackMethod, Func`3 callShouldRetry, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.BackoffRetryUtility`1.ExecuteAsync(Func`1 callbackMethod, IRetryPolicy retryPolicy, CancellationToken cancellationToken, Action`1 preRetryCallback)
   at Microsoft.Azure.Documents.Client.DocumentClient.CreateDocumentCollectionIfNotExistsPrivateAsync(Uri databaseUri, DocumentCollection documentCollection, RequestOptions options)
   at Orleans.Reminders.CosmosDB.CosmosDBReminderTable.TryCreateCosmosDBResources()
   at Orleans.Reminders.CosmosDB.CosmosDBReminderTable.Init()
   at Orleans.OrleansTaskExtentions.WithTimeout(Task taskToComplete, TimeSpan timeout, String exceptionMessage)
   at Orleans.Runtime.ReminderService.LocalReminderService.Start()
   at Orleans.Runtime.Scheduler.AsyncClosureWorkItem.Execute()
   at Orleans.OrleansTaskExtentions.WithTimeout(Task taskToComplete, TimeSpan timeout, String exceptionMessage)
   at Orleans.Runtime.Silo.<OnActiveStart>g__StartReminderService|74_0()
   at Orleans.Runtime.Silo.StartAsyncTaskWithPerfAnalysis(String taskName, Func`1 task, Stopwatch stopWatch)
   at Orleans.Runtime.Silo.OnActiveStart(CancellationToken ct)
   at Orleans.Runtime.SiloLifecycleSubject.MonitoredObserver.OnStart(CancellationToken ct)
   at Orleans.LifecycleSubject.WrapExecution(CancellationToken ct, Func`2 action)
   at Orleans.LifecycleSubject.OnStart(CancellationToken ct)
warn: Orleans.Runtime.Silo[100220]
galvesribeiro commented 5 years ago

Could you please point me what is this "shared" throughput?

I'm not aware of that...

dealproc commented 5 years ago

Newer setting within CosmosDB, and maybe I'm saying the wrong thing, whereas you can purchase RUs at the database level instead of the collection level. In this use-case, you are able to share the RUs across multiple collections, which is a great thing for just getting started, as it makes CosmosDB more cost-effective initially.

Take a look at: https://azure.microsoft.com/en-us/pricing/details/cosmos-db/ under the header Provisioning at different granularities

galvesribeiro commented 5 years ago

Okey, now I understand...

Lets discuss on the PR then...

galvesribeiro commented 5 years ago

Like I said on the PR, I'll investigate further what is happening with that collection and the new RU allocation mode as soon as I have Orleans 2.2 out.

Will keep this issue open to follow up later on as so far, I (and all other users of those packages) are not having those issues.