search

microsoft / azure-load-testing

MIT License
22 stars 2 forks source link

Pipeline runs successfully but does not publish results (pipeline) or create a test run in Azure Load Testing #90

Closed johnAgilisys closed 2 years ago

johnAgilisys commented 2 years ago

Describe the bug The test results are not published in the CI/CD pipeline. Secondly a test run is also not generated in Azure Load Testing

To Reproduce Steps to reproduce the behavior:

  1. Run the pipeline
  2. Step execute successfully and runs the load test
  3. Test reults arent published to the CI/CD Pipeline
  4. Test Run is not generated in the Azure Load Testing (Azure Portal)

Alternatively

  1. Go to Azure portal and open the Azure Load Testing service
  2. Create a Test plan and upload the Jmeter file
  3. run the Test
  4. No test run is generated in the test plan

Expected behavior I should atleast see a test run generate in the Azure Portal

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

Previous Successful Run image

Current Failed result image

AB#1567518

LisaKemeling commented 2 years ago

I had this issue yesterday as well. Also tried a new test and it got stuck while provisioning and failed after about 10 minutes. At some point I got it to run successfully but when trying a new run it succeeded but did not actually run in the portal. Also existing tests that used to run successfully, failed or succeeded in the pipeline but did not actually run.

Is there any way to get insights on the availability of Azure Load Testing and/or are there logs to look at (besides the jmeter logs).

johnAgilisys commented 2 years ago

The pipeline is generating results for the test run. What was the issue as I havent changed anything for it to work @ninallam

ninallam commented 2 years ago

@johnAgilisys @LisaKemeling thanks for reaching out. There were some issues with provisioning in the service during this time. That could be the reason for failure. Can you share the test id so that we can investigate this further? This can be found in the browser URL when you navigate to the test.

LisaKemeling commented 2 years ago

testRunId/8fe13a8d-06f1-4b53-abfd-6680b4f1a04e

That is the only example I have left. The rest I deleted yesterday already. In this example I only added server side components, but it did not pass the provisioning and then failed. I could not run it via the pipeline after this as well.

johnAgilisys commented 2 years ago

@ninallam -The issue is resolved now and is working as expected.. As you can see in the screenshot it did not provision anything and no test run was created in Azure Load testing. It did this on 8 runs up until this morning.

image

LisaKemeling commented 2 years ago

The issue @johnAgilisys mentions seems to be something else indeed, I had that also yesterday. I just ran one of the tests again and now it works fine.

ninallam commented 2 years ago

@radhikajain99 can you please check why does the pipelines task report success even when the test run is not created?

radhikajain99 commented 2 years ago

@ninallam We were not failing the pipeline task in case of validation timeout. The fix for this has been deployed