[Feature Request] Load test configuration yaml file not supporting variable to run Azure Load testing task via Azure Pipeline

microsoft / azure-load-testing

MIT License

22 stars 3 forks source link

[Feature Request] Load test configuration yaml file not supporting variable to run Azure Load testing task via Azure Pipeline #39

Open agrawalamit31 opened 2 years ago

agrawalamit31 commented 2 years ago

Is your feature request related to a problem? Please describe. We are having four applications which we want to load test via single Azure pipeline so tried to create variables in load test yaml file but it didn't work for me. To make it working created 4 separate yaml files corresponding to each service.

Describe the solution you'd like Created below load test config yaml file

variables:
- template: variables.yaml 

version: v0.1
testName:  ${{ testname }}
testPlan: Notification.jmx
description: 'Notification Load Test Run'
engineInstances: 1
secrets:
  - name: notificationAuthCode
    value: https://abcd.azure.net/secrets/authCode

Created varaible.yml file

variables:
  testname: 'TestName'

Azure Load Testing execution task failed.

Please provide a way where we can make our load test config yaml file configurable

AB#1465662

agrawalamit31 commented 2 years ago

@abranj1219 - FYI

ninallam commented 2 years ago

@agrawalamit31 thanks for the feedback. We will explore this option to improve the experience.

Kritika1994 commented 1 year ago

Need a way to run multiple load tests at a scheduled time each day. Today we can't do it with powershell or anything else. Also while configuring yaml we have to write 1 yaml per test even if the configuration is exactly the same. Need a way to bypass that

balajti-bucin-ioan commented 1 year ago

is there any news about this? I am also trying to run multiple JMX files from 1 YAML but I can't :( or is there a workaround? like providing a list of JMX files to run from 1 single YAML file?

ninallam commented 1 year ago

@keysmaster86 A YAML file maps to a test. Currently we allow only 1 JMX per test. Running multiple JMX files is not supported.

BerghammerBN commented 1 year ago

making the config file parameterizable would make a lot of sense. I'd like to be able to:

set the subnet as a parameter, so that I can reuse the config file throughout all stages (different subscriptions)
set the number of engine instances via the pipeline, only than I can manually run various load scenarios
set the failure criteria as a parameter
set the test id

e.g.,

version: v0.1
testId: MyNewTest-${{ parameters.stage }}
displayName: Load Test EventStore API GetDeliverableItemEvents
testPlan: ../scripts/EventStoreAPI_GetDeliverableItemEvents.jmx
description: load testing the EventStore API GetDeliverableItemEvents
engineInstances: ${{ parameters.engineInstances }}
subnetId: "${{ parameters.subnet }}"
failureCriteria:
  - GetDeliverableItemEvents: p90(latency) > ${{ parameters.latencyFailureCriteria }}
  - percentage(error) > {{ parameters.errorFailureCriteria }}