Update all unit tests to work within the runtime environment

google / testrun

A tool to automate verification of network-based device behavior

Apache License 2.0

23 stars 9 forks source link

Update all unit tests to work within the runtime environment #611

Closed jhughesbiot closed 1 month ago

jhughesbiot commented 1 month ago

Unit tests were not guaranteed to run within the same environment as they would in runtime so this updates the unit test structure to use containers as much as possible as well as the venv runtime for all non containerized tests.

Update all test module unit tests to run from within the built container
- Update all host level unit tests to run within the venv
- Move some report tests out of test containers
- Misc unit test cleanup

jboddey commented 1 month ago

I cannot seem to run these unit tests. This is the error I get:

/usr/bin/python3: can't find 'main' module in '/testing/unit/conn/module_test.py' /usr/bin/python3: can't find 'main' module in '/testing/unit/dns/module_test.py' /usr/bin/python3: can't find 'main' module in '/testing/unit/ntp/module_test.py' /usr/bin/python3: can't find 'main' module in '/testing/unit/protocol/module_test.py' /usr/bin/python3: can't find 'main' module in '/testing/unit/services/module_test.py' ^Cdocker: Error response from daemon: OCI runtime start failed: cannot start a container that has stopped: unknown. testing/unit/run.sh: line 64: venv/bin/activate: No such file or directory

Could you quickly document in testing/unit/README.md how to run the unit tests?

jhughesbiot commented 1 month ago

I'll add some docs around it's usage. Were you running the unit test from the testing/unit directory? That's currently required so might be a pathing issue if it was run from somewhere else.

jboddey commented 1 month ago

I'm still getting some errors when running from the correct path: https://paste.googleplex.com/6099375204073472 Also, could we modify the run script to default from running from the root directory?

jhughesbiot commented 1 month ago

Based on the last error for the TLS tests, it looks like you didn't' rebuild the containers since it's missing a new dependency added in the tls test module used for testing.

The other errors appear to be because the binary files are corrected during normal test module startup which is short circuited in testing. Had to move line endings and executable command into test module docker files for consistency to correct this.

Unit test run.sh script is now designed to be executed from the Testrun root directory as well.