search

cyrusimap / cassandane

Other
6 stars 11 forks source link

don't count filtered tests as "OK" in report #81

Open elliefm opened 4 years ago

elliefm commented 4 years ago 
$./testrunner.pl ClamAV
Cyrus::ClamAV.aaasetup                                                  [  OK  ]
Cyrus::ClamAV.notify_deleted                                            [  OK  ]
Cyrus::ClamAV.custom_notify_deleted                                     [  OK  ]

Time:  0 wallclock secs ( 0.01 usr  0.01 sys +  0.31 cusr  0.12 csys =  0.45 CPU)

OK (4 tests)

In this case ClamAV.remove_infected_slow was skipped due to a filter, but the report at the end still counts it toward those "4 tests"

elliefm commented 4 years ago

I think this is just inherent to the way Cassandane blurs the line between what's a test and what's a suite. Test::Unit::Result increments the run_count when the test is started, and Test::Unit::TestSuite doesn't start tests that match the filter, but:

https://github.com/cyrusimap/cassandane/blob/b30e9306093e7df950c2fb5f352cc3766ba09d25/Cassandane/Unit/TestPlan.pm#L844-L847

Uh oh!