search

wargio / naxsi

NAXSI is an open-source, high performance, low rules maintenance WAF for NGINX
GNU General Public License v3.0
305 stars 38 forks source link

Run unit tests on Windows CI #39

Closed staticlibs closed 2 years ago

staticlibs commented 2 years ago

This patch enables running the test suite on Windows CI. Instead of using Test::Nginx::Socket a python script is added that can parse and run the tests.

Note, the runner script is currently very basic, it harcodes the port, doesn't allow test selection, doesn't support request timeouts etc. It also doesn't fail the CI run on test failure - only prints the run summary in the end. The whole test suite run takes about 5 minutes.

I am currently getting the following results:

All: 552
Successful: 546
Skipped: 2
Failed: 4
Errors: 0

Where skipped are tests with [[ ... ]] request eval syntax, and failures seem to be caused by runner script shortcomings in parsing request eval multipart bodies (don't look like real failures to me, going to revisit).

The idea is to keep these test runs in info-only mode for some time until the runs are stable enough.

wargio commented 2 years ago

Probably just rewriting the unit tests in python would be much simpler and more portable.

staticlibs commented 2 years ago

I think I can generate the python unit tests from existing test files. The runner script is already doing the major part of this job, just instead of running the parsed tests directly it needs to generate a python function for every test. I will experiment with this and return in a few days.

wargio commented 2 years ago

if you could do that, it would be awesome. Please use the least amount of libraries possible.