Use ARM Templates for Benchmarking

[NaluTripician]

Background

The goal of this feature is to make the process of using the CosmosDB benchmarking tools as easy as possible for our customers. Here we are looking to use ARM templates to deploy and run our benchmarking.

Design Document

This document #3896 describes the end to end design in detail.

This will be done in two stages:

Stage 1

First, uses an ARM template that deploys the Azure Container Instance based model our benchmarking currently uses. This will also add one click deploy functionality.

• PR: https://github.com/Azure/azure-cosmos-dotnet-v3/pull/3838

Stage 2

The next stage will take that idea and evolve it further. Here we will switch to a VM based model which aims to simplify the process and reduce costs. Here we will also add more features:

• Ability to execute benchmarking on multiple machines with one ARM Template #3888
• Include more input parameters for testing features such as ClientTelemetry and DistributedTracing
• Include extensive logs with full diagnostics #3889
• AppInsights: Live metrics (10s) #3892
  • Success rate
  • Error
  • P90, P99, P999, P9999
• CTL+ ARM Integration (future work)
• Add ARM template in benchmarking repo
• Instructions on how to change config when benchmark is running

Other Resources:

https://learn.microsoft.com/en-us/azure/cosmos-db/nosql/benchmarking-framework?tabs=azure-portal Benchmarking Repo: https://github.com/Azure/azure-db-benchmarking/tree/main

[philipthomas-MSFT]

:) Friendly reminder to fill in description.