search

testomatio / reporter

Testomatio Reporter
126 stars 22 forks source link

Reported tests could not be matched to tests in the project if TESTOMATIO_CREATE option used #223

Closed XavierDeMontis closed 12 months ago

XavierDeMontis commented 1 year ago

Describe the bug We run our tests on the CI. Those tests are imported into Testomat using the appropriate commands (as instructed in the doc). But in the end no test can be matched, which dramatically lowers Testomat abilities.

To Reproduce

  1. We run our tests on the CI with the following command : TESTOMATIO_RUN=${run} TESTOMATIO_CREATE=1 TESTOMATIO=${testomatio} TESTOMATIO_TITLE=${STARTED_BY} TESTOMATIO_ENV="${BRANCH_NAME}, ${LINKURIOUS_DB_VENDOR}" codeceptjs run --debug --verbose --grep "^(?=.*${grep})(?!.*@Skipped).*" --reporter mocha-multi-reporters --reporter-options
  2. When the run is over, none of my tests match and I see the following message image

Expected behavior Even if all my tests do not match I expect TESTOMATIO_CREATE=1 to do what it's supposed to do (automatically create missing tests).

Screenshots If applicable, add screenshots or video to help explain your problem.

Desktop (please complete the following information):

Additional context Add any other context about the problem here.

olexandr13 commented 1 year ago

@XavierDeMontis please specify the version of reporter and codeceptjs you use

XavierDeMontis commented 1 year ago

Hi @olexandr13 If I'm not mistaken we have codeceptjs 3.2.2 mocha-multi-reporter 1.5.1

olexandr13 commented 1 year ago

Hi @olexandr13 If I'm not mistaken we have codeceptjs 3.2.2 mocha-multi-reporter 1.5.1

See the problem on our side. Will let you know when fixed. Thanks for reporting.

olexandr13 commented 1 year ago

@DavertMik backend processes create=true option in test data incorrectly. and tests are not included into report

olexandr13 commented 1 year ago

have a call

olexandr13 commented 1 year ago

Fixed. Please check if reproduced for you on the latest reporter version @XavierDeMontis

TetianaKhomenko commented 12 months ago

@XavierDeMontis please let us know if the issue was fixed on your side

XavierDeMontis commented 12 months ago

Hi @TetianaKhomenko we're currently upgrading our dependecies, but since it did not happen for a loooong time it will take quite some time to my devs. In the mean time I think you can consider it fixed and if, once updated, we still see an issue I'll open a new ticket.

Regards