search

microsoft / playwright-testing-service

MIT License
78 stars 13 forks source link

[BUG]: Log Analytics receives no monitoring data after a test run #57

Open webash opened 11 months ago

webash commented 11 months ago

Describe the bug Running tests (which execute successfully, and are visible in the portal) does not produce any entries in the Log Analytics workspace.

To Reproduce Steps to reproduce the behavior:

  1. Create MPT workspace
  2. Navigate to the workspace in Azure
  3. Enable Diagnostics Settings by adding a rule to push all entries to a Log Analytics workspace
  4. Run tests against the workspace
  5. See the tests have displayed/finished in the MPT portal
  6. Check Log Analytics workspace - all tables are empty

Expected behavior Per the monitoring and reference pages, the Log Analytics workspace should have entries in the Operation logs after test runs.

Screenshots image

Setup information (please complete the following information):

Additional context Add any other context about the problem here.

AB#1899173

vvs11 commented 11 months ago

Thanks @webash for raising this. This is a bug in the service. We will fix it.

Could you help us understand your use-case here?

webash commented 11 months ago

Ultimately we're trying to do something that Ceruleoscope offered, but no longer seems to work (and appears to be abandonware): Use Playwright to run tests against our websites, and measure their performance and availability. If metrics drop below intended thresholds, we can trigger an incident and begin work to raise performance (or fix an unavailable instance).

So the Log Analytics stats from Playwright Testing service, we want to see whether they will at least give us a basic total elapsed time metric, which will be enough for our early phases of these performance metrics.

webash commented 11 months ago

Any idea when this might be fixed? Is there anything I can do my side to workaround it for now?

vvs11 commented 10 months ago

@webash, Apologies for the late response

We are working on a fix for this but for now, only CRUD operations related to Access Tokens will be visible in log analytics. We will have to come up with a better solution for your use-case. We are currently collecting feedback on the test reporting feature: https://github.com/microsoft/playwright-testing-service/issues/47

If this is something that will solve your use-case, please upvote this and add your use-case in comments. It will help us prioritize the features and build a better solution.

Thanks

MattRaetz commented 2 weeks ago

Is there any update on this? This functionality is listed in the documentation but we are seeing the same behavior of no data in our log analytics workspace

vvs11 commented 1 week ago

Hi @MattRaetz Could you help us understand what data you want to see here? Will the reporting feature mentioned in #47 solve your use-case? Please let us know.