search

marklogic-community / marklogic-unit-test

Automated testing for MarkLogic
https://marklogic-community.github.io/marklogic-unit-test/
Other
13 stars 25 forks source link

Nested test suites may lead to incorrect setup module selection #204

Open dmcassel opened 10 months ago

dmcassel commented 10 months ago

I have a test suite suiteA that has both setup.sjs and suiteSetup.sjs. Nested within suiteA, I had a subdirectories for a related test: verify. This subdirectory had its own suiteSetup.sjs. This worked fine.

I added a new subdirectory remind that has a suiteSetup.sjs. Now when I run suiteA, tests fail because instead of running suiteA/suiteSetup.sjs, it's running suiteA/remind/suiteSetup.sjs.

The suite setup module is identified in the controller by calling test:list-from-database and taking the first module that passes is-suite-setup-module(.). It looks like the modules came back in alphabetical order (document order?) and so suiteA/suiteSetup.sjs was selected before suiteA/verify/suiteSetup.sjs, but suiteA/remind/suiteSetup.sjs messed up the order.

The fix may be as simple as changing the cts:directory-query and xdmp:directory calls from "infinity" to "1", but I haven't tested that.

dmcassel commented 10 months ago

Workaround: de-nest the test suites.