search

ansible / vscode-ansible

vscode/vscodium extension for providing Ansible auto-completion and integrating quality assurance tools like ansible-lint, ansible syntax check, yamllint, molecule and ansible-test.
https://ansible.readthedocs.io/projects/vscode-ansible/
MIT License
360 stars 86 forks source link

Code coverage support in UI test #1346

Closed TamiTakamiya closed 3 months ago

TamiTakamiya commented 3 months ago

This is for adding code coverage support to our UI tests and enable them in CI. This uses the new codo coverage support implemented on vscode-extension-tester.

Usage:

yarn install
yarn webpack-dev
yarn coverage-ui-with-mock-lightspeed-server

The html report is generated in out/coverage/ui-test. A sample output is shown as follows: image

Description:

This PR changes the location of lcov.info files. In a CI build, matrix names "test" and "test (macos)" will produce three lcov.info files:

  1. unit --> out/coverage/unit/lcov.info
  2. ui-test --> out/coverage/ui-test/lcov.info
  3. e2e --> out/coverage/e2e/lcov.info

These files are uploaded with codecov-action and merged into one report on the Codecov web UI.

For example the code coverage of playbookGeneration.ts is about 94% when these results are combined: image

However, if only the e2e test result is selected, image

The coverage goes down to about 11%: image