[Documentation Request] Performance Testing Examples

[PeterConijn]

Is your feature request related to a problem? Please describe. We have our pipelines set up to build and run automated tests, and now we want to start integrating performance tests. We are can create the test codeunits, but we are struggling on how to get this into a pipeline

Describe the solution you'd like We would very much like to have an example or two in the Examples folder that includes running performance tests and getting the results into a good format.

Describe alternatives you've considered I have Googled this, but information on BCPT in pipelines is scarce.

[waldo1001]

We have a new step on the shelf, which is about to be released - most likely this week or next week.

[waldo1001]

In the next version, you will be able to run BCPT - we'd love to have some feedback on that.

About the results: I think it makes most sense to capture the results in telemetry. That way, you can have metrics, powerbi, graphs, regression overview .. all you wish for. Would that work for you?

[JohnnyUndercover]

having the same problem at the moment, i'am struggling setting the application insights key for the docker or should i set the insights key for my custom bcpt app? I also tried getting something exported with export_results, but didn't know where to look for the results... would be really helpfull to have an example :)

[waldo1001]

I'm working on an example, but here's a preliminary one (remark: still work in progress)

name: $(Build.BuildId)

trigger: none

variables:
  # - group: 'Secrets'  # link to a variable group that has secret variables, like URL to license file, or sasToken for insider builds
  - name: bcpt_appinsights
    value: "InstrumentationKey=xyz-a16c-4b09-8c39-abc;IngestionEndpoint=https://westeurope-5.in.applicationinsights.azure.com/;LiveEndpoint=https://westeurope.livediagnostics.monitor.azure.com/"

pool: WaldoHetzner

steps:
  - checkout: self
    clean: true

  - task: ALOpsDockerCreate@1
    displayName: "ALOps - Create Docker Image if necessary" # It will reuse the image if already exists, or build one if it doesn't.
    inputs:
      artifacttype: Sandbox
      versionselect: Weekly
      artifactcountry: base

  - task: ALOpsDockerStart@1
    displayName: "ALOps - Start Docker Container" # No need to provide any details - it will get the details from previous step
    inputs:
      docker_pull: false
      docker_parameters: |
        --env customNavSettings=EnableTaskScheduler=false

  - task: ALOpsDockerWait@1
    displayName: "ALOps - Wait for Docker Container to start"
    inputs:
      search_string: "Ready for connections!"

  #Install Test-apps - included in artifact, so easy to reference the names of the apps - ALOps will find them
  - task: ALOpsAppPublish@1
    displayName: "ALOps - Install AL TestTool" # install necessary dependent apps for testability
    inputs:
      usedocker: true
      installaltesttool: true
      skip_verification: true
      install_al_app_names: |
        Tests-TestLibraries
        System Application Test
        System Application Test Library
        Any
        Library Assert
        Test Runner
        Permissions Mock
        Library Variable Storage
        Performance Toolkit

  - task: ALOpsAppCompiler@1
    displayName: "ALOps - Compile Extension: App"
    inputs:
      usedocker: true
      nav_app_version: "?.?.*.0" # ? = Take digit from the app.json, * = buildnumber
      targetproject: "app/app.json" # The App
      app_file_suffix: "_APP" # A trick to uniquely identify the APP (from the TEST-app)

  - task: ALOpsAppPublish@1
    displayName: "ALOps - Publish Extension"
    inputs:
      usedocker: true
      nav_artifact_app_filter: "*.app"
      skip_verification: true

  - task: ALOpsAppCompiler@1
    displayName: "ALOps - Compile Extension: Test-App"
    inputs:
      usedocker: true
      nav_app_version: "?.?.*.0"
      targetproject: "test/app.json"
      app_file_suffix: "_TEST"

  - task: ALOpsAppPublish@1
    displayName: "ALOps - Publish Extension"
    inputs:
      usedocker: true
      nav_artifact_app_filter: "*.app"
      skip_verification: true

  - task: ALOpsAppCompiler@1
    displayName: "ALOps - Compile Extension: BCPT-App"
    inputs:
      usedocker: true
      nav_app_version: "?.?.*.0"
      targetproject: "BCPT/app.json" # The Test-App
      app_file_suffix: "_TEST" # A trick to uniquely identify the Test-App (from the App)

  - task: ALOpsAppPublish@1
    displayName: "ALOps - Publish Extension"
    inputs:
      usedocker: true
      nav_artifact_app_filter: "*.app"
      skip_verification: true

  - task: ALOpsAppCompiler@1
    displayName: "ALOps - Compile Extension: MoreTelemetry-App"
    inputs:
      usedocker: true
      nav_app_version: "?.?.*.0"
      targetproject: "MoreTelemetry/app.json" # The Test-App
      app_file_suffix: "_TEST" # A trick to uniquely identify the Test-App (from the App)

  - task: ALOpsAppPublish@1
    displayName: "ALOps - Publish Extension"
    inputs:
      usedocker: true
      nav_artifact_app_filter: "*.app"
      skip_verification: true

  - task: ALOpsDockerExec@1
    displayName: "ApplicationInsightsConnectionString"
    inputs:
      inline_script: |
        Set-NAVServerConfiguration -ServerInstance "BC" `
                                   -KeyName  EnableLockTimeoutMonitoring  `
                                   -KeyValue $true

        Set-NAVServerConfiguration -ServerInstance "BC" `
                                   -KeyName  EnableDeadlockMonitoring  `
                                   -KeyValue $true

        Set-NAVServerConfiguration -ServerInstance "BC" `
                                   -KeyName  SqlLongRunningThresholdForApplicationInsights  `
                                   -KeyValue 500

        Set-NAVServerConfiguration -ServerInstance "BC" `
                                   -KeyName  ALLongRunningFunctionTracingThresholdForApplicationInsights  `
                                   -KeyValue 10000

        Set-NAVServerInstance -ServerInstance "BC" -Restart

        Set-NAVServerConfiguration -ServerInstance "BC" -KeyName "ApplicationInsightsConnectionString" -KeyValue "$(bcpt_appinsights)" -ApplyTo All

  - task: ALOpsDockerExec@1
    displayName: Install MSAL.PS
    inputs:
      inline_script: |
        install-packageprovider -name NuGet -MinimumVersion 2.8.5.201 -Force
        install-module MSAL.PS -force -confirm:$true

  - task: ALOpsInfo@1

  - task: ALOpsBCPT@1
    displayName: "Run BC Performance Tests"
    inputs:
      usedocker: true
      export_results: true
      suite_code: BCPT

  - task: ALOpsDockerRemove@1
    displayName: "ALOps - Remove Docker Container"
    enabled: true
    condition: always()

[PeterConijn]

That is amazing news!! Using telemetry would absolutely work for us.

I do notice in your preliminary example above that both the "regular" test app and the BCPT app have the app_file_suffix "_TEST"; is that not going to bite each other?

For more information on BCPT:

• Stefano Dimiliani has a good article on setting up the BCPT here
• The Microsoft Docs are very informative on explaining how to set up scenarios, but some practical .
• Bert Verbeek has a good article on best practices here

However, I do know that you (@waldo1001) have experience on setting these up too, so any additional advice would be greatly appreciated.

[waldo1001]

As said - it's very temp and should be simplified a lot .. I just can't focus on it for now .. will try tomorrow or next week to clean up this example...