search

retis-org / retis

Tracing packets in the Linux networking stack & friends
https://retis.readthedocs.io/en/stable/
100 stars 14 forks source link

Add Funtional tests #135

Closed amorenoz closed 1 year ago

amorenoz commented 1 year ago

This PR adds two things: 1) Using cirrus-ci, and Vagrant, run tests on a controlled VM (currently only Fedora 36, Centos9 and Ubuntu Jammy). These tests are activated using the github label.

2) Using pytest (a well-known python testing framework), add a small framework to help write tests. A small test is created as demonstration. It creates a ovs bridge with two veths and two netns, runs a ping while retis is collecting events. Then the events are read and asserted.

amorenoz commented 1 year ago

I have not added functional tests yet (just a dummy one). I'll leave that for the PRs that actually touch functionality or other tests-only PRs. Rebased on top of @vlrpl 's fix of rust 1.65 and added Centos Stream 9 and Ubuntu Jammy to the test matrix

amorenoz commented 1 year ago

The current behavior is: 1) PRs without run-functional-tests label: Only unit tests are run. The rest return "neutral" result. Can be triggered manually clicking in Details -> Trigger 2) PRs with run-functional-tests label: All tests are run. Note that changing the label does not re-trigger tests in github. So the label will make effect the next time the PR is chenged. Manual triggering is still possible as per 1) 3) For all non-PR branches. Only Unit tests, manually triggering the functional tests is possible via cirrus-ci web UI (same as 1)

atenart commented 1 year ago 
1. PRs without `run-functional-tests` label: Only unit tests are run. The rest return "neutral" result. Can be triggered manually clicking in `Details` -> `Trigger`

So this the restrictions defined on functional_task: do not apply I guess?

2. PRs with `run-functional-tests` label: All tests are run. Note that changing the label does not re-trigger tests in github. So the label will make effect the next time the PR is chenged. Manual triggering is still possible as per 1)

Adding the label and requiring a manual trigger is somehow error-prone. But I guess we don't really have a choice if we want those tests to run automatically for next versions.

amorenoz commented 1 year ago 
1. PRs without `run-functional-tests` label: Only unit tests are run. The rest return "neutral" result. Can be triggered manually clicking in `Details` -> `Trigger`

So this the restrictions defined on functional_task: do not apply I guess?

What do you mean? functional_task is only run if:

  required_pr_labels: run-functional-tests
  only_if: "$CIRRUS_PR != ''"

so for PRs that do not have the label, it's not run.

atenart commented 1 year ago 
1. PRs without `run-functional-tests` label: Only unit tests are run. The rest return "neutral" result. Can be triggered manually clicking in `Details` -> `Trigger`

So this the restrictions defined on functional_task: do not apply I guess?

What do you mean? functional_task is only run if:

  required_pr_labels: run-functional-tests
  only_if: "$CIRRUS_PR != ''"

Initial quoting was bad. I meant if we manually trigger a runtime test, should the PR have the label first or will it run regardless? I guess the tests would be skipped?

amorenoz commented 1 year ago

Initial quoting was bad. I meant if we manually trigger a runtime test, should the PR have the label first or will it run regardless? I guess the tests would be skipped?

If you manually trigger the test, it's run regardless of branch name or PR labels.