search

tangle-network / relayer

🕸️ The Webb Relayer Network
https://webb-tools.github.io/relayer/
Apache License 2.0
22 stars 13 forks source link

Run CI integration tests in parallel #458

Closed Nutomic closed 1 year ago

Nutomic commented 1 year ago

Summary of changes

Reference issue to close (if applicable)

Code Checklist

Nutomic commented 1 year ago

This is odd, evm integration tests are failing because of an assertion failure in Vanchor Transaction relayer, and for some reason substrate integration tests are cancelled automatically before starting every time. Not sure what the problem might be, maybe there is some hidden dependency between different tests?

shekohex commented 1 year ago

This is odd, evm integration tests are failing because of an assertion failure in Vanchor Transaction relayer, and for some reason substrate integration tests are cancelled automatically before starting every time. Not sure what the problem might be, maybe there is some hidden dependency between different tests?

I guess it is because of the fail-fast

Nutomic commented 1 year ago

After a couple of restarts everything is passing. Time went down from 30m to 20m. One test is still listed as pending because I removed it (it wasnt doing anything). Not sure how to get rid of that one.

I guess it is because of the fail-fast

Ah makes sense. Still weird that evm tests failed.

shekohex commented 1 year ago

We should close this as it is not feasible right now, until we implement #391

Nutomic commented 1 year ago

It looks like you linked the wrong issue, that one was closed a year ago. And I dont see why this wouldnt be feasible, its only grouping the integration tests differently for CI. And as far as I can tell its already working, only problem is that linux-integration-tests (stable, aarch64-unknown-linux-musl) is marked as mandatory, but I removed it because its not doing anything.

shekohex commented 1 year ago

Updated my comment with the right one. For that I only got the sense that it is not gonna be feasible from your comments. feel free to re-open this PR if you think otherwise.

Nutomic commented 1 year ago

Okay reopened. I think it should work fine if you or a repo admin removes linux-integration-tests (stable, aarch64-unknown-linux-musl) from https://github.com/webb-tools/relayer/settings/branch_protection_rules/

salman01zp commented 1 year ago

We can remove linux-integration-tests (stable, aarch64-unknown-linux-musl) from the mandatory test for develop branch. @shekohex @dutterbutter

shekohex commented 1 year ago

We can remove linux-integration-tests (stable, aarch64-unknown-linux-musl) from the mandatory test for develop branch. @shekohex @dutterbutter

These are the only required checks in develop branch: image

Nutomic commented 1 year ago

This PR removes the check linux-integration-tests (stable, aarch64-unknown-linux-musl) and adds two separate ones instead:

This means that the same tests are run, but on two separate machines so that it can be in parallel. Merging this PR requires removing linux-integration-tests (stable, aarch64-unknown-linux-musl) from mandatory checks and adding the two new ones instead.