search

SpamExperts / OrangeAssassin

OrangeAssassin
https://orangeassassin.org
Apache License 2.0
59 stars 19 forks source link

Tests are very noisy #33

Open tonyandrewmeyer opened 8 years ago

tonyandrewmeyer commented 8 years ago

There's a lot of (logging) noise when running the tests:

root@server1:~/SpamPAD# env PYTHONPATH=. python tests/__init__.py
............................................................................................................................................................................[]
.Config: no rules were found!
.
.1 message(s) examined
.1 message(s) examined
................................................................................................................................2016-02-11 10:43:21,978 [3103] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v310.pre
2016-02-11 10:43:21,978 [3103] WARNING Plugin not available: Mail::SpamAssassin::Plugin::Check
2016-02-11 10:43:21,978 [3103] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v320.pre
2016-02-11 10:43:21,978 [3103] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/20.cf
2016-02-11 10:43:21,979 [3103] DEBUG Adding rule GTUBE with: {u'describe': u'Generic Test for Unsolicited Bulk Email', 'type': u'body', u'score': u'1000', 'value': u'/XJS\\*C4JDBQADN1\\.NSBN3\\*2IDNEN\\*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL\\*C\\.34X/'}
2016-02-11 10:43:21,979 [3103] INFO 1 rules loaded
2016-02-11 10:43:21,979 [3103] DEBUG Checked rule * 1000.0 GTUBE BODY: Generic Test for Unsolicited Bulk Email: True
.2016-02-11 10:43:22,419 [3106] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v310.pre
2016-02-11 10:43:22,419 [3106] WARNING Plugin not available: Mail::SpamAssassin::Plugin::Check
2016-02-11 10:43:22,419 [3106] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v320.pre
2016-02-11 10:43:22,419 [3106] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/20.cf
2016-02-11 10:43:22,420 [3106] DEBUG Adding rule TEST_RULE with: {'type': u'body', 'value': u'/abcd/'}
2016-02-11 10:43:22,420 [3106] INFO 1 rules loaded
2016-02-11 10:43:22,420 [3106] DEBUG Checked rule * 1.0 TEST_RULE BODY: No description available.: True
.2016-02-11 10:43:22,862 [3109] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v310.pre
2016-02-11 10:43:22,863 [3109] WARNING Plugin not available: Mail::SpamAssassin::Plugin::Check
2016-02-11 10:43:22,863 [3109] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v320.pre
2016-02-11 10:43:22,863 [3109] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/20.cf
2016-02-11 10:43:22,863 [3109] DEBUG Adding rule TEST_RULE with: {'type': u'body', 'value': u'/one-two-three/'}
2016-02-11 10:43:22,863 [3109] INFO 1 rules loaded
2016-02-11 10:43:22,864 [3109] DEBUG Checked rule * 1.0 TEST_RULE BODY: No description available.: True
.2016-02-11 10:43:23,311 [3112] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v310.pre
2016-02-11 10:43:23,312 [3112] WARNING Plugin not available: Mail::SpamAssassin::Plugin::Check
2016-02-11 10:43:23,312 [3112] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v320.pre
2016-02-11 10:43:23,312 [3112] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/20.cf
2016-02-11 10:43:23,312 [3112] DEBUG Adding rule TEST_RULE with: {'type': u'body', 'value': u'/abcd/'}
2016-02-11 10:43:23,312 [3112] INFO 1 rules loaded
.2016-02-11 10:43:23,752 [3115] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v310.pre
2016-02-11 10:43:23,752 [3115] WARNING Plugin not available: Mail::SpamAssassin::Plugin::Check
2016-02-11 10:43:23,753 [3115] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v320.pre
2016-02-11 10:43:23,753 [3115] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/20.cf
2016-02-11 10:43:23,753 [3115] DEBUG Adding rule TEST_RULE with: {'type': u'body', 'value': u'/abcd/'}
2016-02-11 10:43:23,753 [3115] INFO 1 rules loaded
.2016-02-11 10:43:24,190 [3118] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v310.pre
2016-02-11 10:43:24,190 [3118] WARNING Plugin not available: Mail::SpamAssassin::Plugin::Check
2016-02-11 10:43:24,190 [3118] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v320.pre
2016-02-11 10:43:24,190 [3118] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/20.cf
2016-02-11 10:43:24,190 [3118] DEBUG Adding rule TEST_RULE with: {'type': u'body', 'value': u'/abcd/'}
2016-02-11 10:43:24,191 [3118] DEBUG Adding rule GTUBE with: {'type': u'body', 'value': u'/XJS\\*C4JDBQADN1\\.NSBN3\\*2IDNEN\\*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL\\*C\\.34X/'}
2016-02-11 10:43:24,191 [3118] INFO 2 rules loaded
2016-02-11 10:43:24,191 [3118] DEBUG Checked rule * 1.0 TEST_RULE BODY: No description available.: True
2016-02-11 10:43:24,191 [3118] DEBUG Checked rule * 1.0 GTUBE BODY: No description available.: True
.2016-02-11 10:43:24,628 [3121] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v310.pre
2016-02-11 10:43:24,628 [3121] WARNING Plugin not available: Mail::SpamAssassin::Plugin::Check
2016-02-11 10:43:24,628 [3121] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v320.pre
2016-02-11 10:43:24,628 [3121] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/20.cf
2016-02-11 10:43:24,628 [3121] DEBUG Adding rule TEST_RULE with: {'type': u'body', 'value': u'/abcd/'}
2016-02-11 10:43:24,629 [3121] INFO 1 rules loaded
2016-02-11 10:43:24,629 [3121] DEBUG Checked rule * 1.0 TEST_RULE BODY: No description available.: True
.2016-02-11 10:43:25,070 [3124] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v310.pre
2016-02-11 10:43:25,070 [3124] WARNING Plugin not available: Mail::SpamAssassin::Plugin::Check
2016-02-11 10:43:25,070 [3124] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v320.pre
2016-02-11 10:43:25,070 [3124] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/20.cf
2016-02-11 10:43:25,070 [3124] DEBUG Adding rule TEST_RULE with: {'type': u'body', 'value': u'/abcd/'}
2016-02-11 10:43:25,070 [3124] INFO 1 rules loaded
2016-02-11 10:43:25,071 [3124] DEBUG Checked rule * 1.0 TEST_RULE BODY: No description available.: True
.2016-02-11 10:43:25,507 [3127] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v310.pre
2016-02-11 10:43:25,508 [3127] WARNING Plugin not available: Mail::SpamAssassin::Plugin::Check
2016-02-11 10:43:25,508 [3127] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v320.pre
2016-02-11 10:43:25,508 [3127] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/20.cf
2016-02-11 10:43:25,508 [3127] DEBUG Adding rule TEST_RULE with: {'type': u'body', 'value': u'/abddd/'}
2016-02-11 10:43:25,508 [3127] INFO 1 rules loaded
2016-02-11 10:43:25,508 [3127] DEBUG Checked rule * 1.0 TEST_RULE BODY: No description available.: False
.2016-02-11 10:43:25,949 [3130] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v310.pre
2016-02-11 10:43:25,950 [3130] WARNING Plugin not available: Mail::SpamAssassin::Plugin::Check
2016-02-11 10:43:25,950 [3130] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v320.pre
2016-02-11 10:43:25,950 [3130] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/20.cf
2016-02-11 10:43:25,950 [3130] DEBUG Adding rule TEST_RULE with: {'type': u'body', 'value': u'/one-two-four/'}
2016-02-11 10:43:25,950 [3130] INFO 1 rules loaded
2016-02-11 10:43:25,951 [3130] DEBUG Checked rule * 1.0 TEST_RULE BODY: No description available.: False
.2016-02-11 10:43:26,394 [3133] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v310.pre
2016-02-11 10:43:26,394 [3133] WARNING Plugin not available: Mail::SpamAssassin::Plugin::Check
2016-02-11 10:43:26,394 [3133] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v320.pre
2016-02-11 10:43:26,394 [3133] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/20.cf
2016-02-11 10:43:26,394 [3133] DEBUG Adding rule TEST_RULE with: {'type': u'body', 'value': u'/abcd/'}
2016-02-11 10:43:26,394 [3133] DEBUG Adding rule GTUBE with: {'type': u'body', 'value': u'/XJS\\*C4JDBQADN1\\.NSBN3\\*2IDNEN\\*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL\\*C\\.34X/'}
2016-02-11 10:43:26,394 [3133] INFO 2 rules loaded
2016-02-11 10:43:26,396 [3133] DEBUG Checked rule * 1.0 TEST_RULE BODY: No description available.: False
2016-02-11 10:43:26,396 [3133] DEBUG Checked rule * 1.0 GTUBE BODY: No description available.: False
.2016-02-11 10:43:26,835 [3136] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v310.pre
2016-02-11 10:43:26,835 [3136] WARNING Plugin not available: Mail::SpamAssassin::Plugin::Check
2016-02-11 10:43:26,835 [3136] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v320.pre
2016-02-11 10:43:26,835 [3136] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/20.cf
2016-02-11 10:43:26,835 [3136] DEBUG Adding rule GTUBE with: {u'describe': u'Generic Test for Unsolicited Bulk Email', 'type': u'body', u'score': u'1000', 'value': u'/XJS\\*C4JDBQADN1\\.NSBN3\\*2IDNEN\\*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL\\*C\\.34X/'}
2016-02-11 10:43:26,836 [3136] INFO 1 rules loaded
2016-02-11 10:43:26,836 [3136] DEBUG Checked rule * 1000.0 GTUBE BODY: Generic Test for Unsolicited Bulk Email: False
.2016-02-11 10:43:27,277 [3139] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v310.pre
2016-02-11 10:43:27,278 [3139] WARNING Plugin not available: Mail::SpamAssassin::Plugin::Check
2016-02-11 10:43:27,278 [3139] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v320.pre
2016-02-11 10:43:27,278 [3139] DEBUG Loading plugin TestPluginReportRevoke from /root/SpamPAD/tests/util/sample_plugin.py
2016-02-11 10:43:27,279 [3139] INFO Plugin TestPluginReportRevoke loaded
2016-02-11 10:43:27,279 [3139] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/20.cf
2016-02-11 10:43:27,279 [3139] INFO 0 rules loaded
.usage: match.py [-h] [-n NICE] [-P] [--show-unknown] [-r | -k] [-D] [-v] -C
                CONFIGPATH --sitepath SITEPATH [-t] [-R]
                [path [path ...]]
match.py: error: argument -k/--revoke: not allowed with argument -r/--report
.2016-02-11 10:43:28,157 [3145] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v310.pre
2016-02-11 10:43:28,158 [3145] WARNING Plugin not available: Mail::SpamAssassin::Plugin::Check
2016-02-11 10:43:28,158 [3145] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/v320.pre
2016-02-11 10:43:28,158 [3145] DEBUG Loading plugin TestPluginReportRevoke from /root/SpamPAD/tests/util/sample_plugin.py
2016-02-11 10:43:28,158 [3145] INFO Plugin TestPluginReportRevoke loaded
2016-02-11 10:43:28,158 [3145] DEBUG Parsing file: /root/SpamPAD/tests/test_match_conf/20.cf
2016-02-11 10:43:28,158 [3145] INFO 0 rules loaded
.
----------------------------------------------------------------------
Ran 319 tests in 7.034s

OK

When the tests are run like this, I should only see a long line of ..

tonyandrewmeyer commented 8 years ago

I was having an unrelated chat with @alexkiro and he mentioned that we're typically running with pytest (my bad) and that automatically hides this. I still think we want to fix this, though. No urgency (but there wasn't any urgency before, either ;)).