search

tarantool / tarantool-qa

QA related issues of Tarantool
3 stars 0 forks source link

debug_coverage workflow can be skipped at all: both push and pull request #147

Closed Totktonada closed 2 years ago

Totktonada commented 2 years ago

It seems, it is clash of https://github.com/tarantool/tarantool/commit/660e3263b9ea661c2adea23fbc905757e3fc79db (where we prefer pull request run even on pull requests from inside the repository) and https://github.com/tarantool/tarantool/commit/6c2e664e4f1419796269f6c3bd3db620acfd1c35 (where we switched it to run on push on such pull requests, but kept the logic for discarding push job if there is a pull request job).

I hope this complex logic for reporting coveralls results may be implemented in some simpler way.

Totktonada commented 2 years ago

For example, push job:

image

Pull request job:

image

I catched it, because observed successful debug_coverage push job with 10 seconds duration.

The commit, where I see it: https://github.com/tarantool/tarantool/commit/e6fccad16f8ccdda8797e9627a8ea5e2e5213a85.

ylobankov commented 2 years ago

It was decided to split the debug_coverage.yml workflow into two independent workflows: debug.yml and coverage.yml.

The first one will be run on push to a branch and creating/updating external pool requests. In other words, it will be run always.

The second one will be run on push to a branch only if the branch has the full-ci suffix (some-branch-full-ci) or the full-ci label is set on a pool request, internal or external - it doesn't matter.

Reasoning:

  1. The coverage testing is heavy testing due to enabling long tests and takes at least 2 times more time than debug build testing. So testing that is done at the first stage (lint, release, debug) will pass faster.

  2. The coverage report as a pool request comment from coveralls.io can be obtained only if a pool request event happened. It is a limitation of the coverallsapp/github-action action [1]. So there is no sense to run the coverage workflow on push events. But the coverage report on push still can be seen on coveralls.io if the branch has the full-ci suffix.

  3. The logic of the debug_coverage.yml workflow is tricky enough and no one understands how it works. So we would like to have an absolutely transparent and simple workflow.

Totktonada commented 2 years ago

The coverage testing is heavy testing due to enabling long tests and takes at least 2 times more time than debug build testing.

Also due to gcov instrumentation, which has its cost.