MailWatch stopped logging in to MySQL

[branko77]

Today my MailWatch just stopped logging in to MySQL. Last Message that has been logged was few hours ago. My postfix is working fine and I can send and receive messages. I received few of messages with this text: "MailScanner was attacked by a Denial Of Service attack, and has therefore deleted this part of the message. Please contact your e-mail providers for more information if you need it, giving them the whole of this report. Attack in: /var/spool/MailScanner/incoming/36423/4335498CC.AB426/nmsg-36423-81.html " After reboot of my server my NAGIOS is reporting that Clamd is critical with error "connect to address 127.0.0.1 and port 3310: Connection refused" When I check MySQL I can see that last message was logged few hours ago and that is not logging. maillog reports that MailWatch: Logging message 5D18798D8.A7796 to SQL but no logged message i maillog. I didn't made any changes what so ever...

[mmgomess]

It does not seem to be a config problem because I have the same settings in ubuntu 14.04 and ubuntu 16.04 but the error only appears in ubuntu 16.

2017-01-27 6:39 GMT-02:00 asuweb notifications@github.com:

Is there a reason you are binding clam to all interfaces?

Try binding clam to the 127.0.0.1 and see if that issue goes away.

It sounds like you might need to do a thorough review of your configuration files to track down the issue. I've always found it an issue upgrading either mailscanner or mailwatch as often there are bits left over or moved in a file that are hard to track down.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mailwatch/1.2.0/issues/430#issuecomment-275614819, or mute the thread https://github.com/notifications/unsubscribe-auth/AJSnk3kl-92ig7TYZJI3NkaxecmGl4Wrks5rWa0vgaJpZM4Lq9fW .

[asuweb]

How should I do that?

In /etc/clamd.conf from memory - search for "bind" and you'll probably see a note above saying it's best to bind it to localhost.

[stefaweb]

The variable is TCPAddr.

By default clamd binds to INADDR_ANY.
This option allows you to restrict the TCP address and provide some degree of protection from the outside world. This option can be  specified multiple times in order to listen on multiple IPs. IPv6 is now supported.
Default: disabled

[ihsankhattak]

Dear All, i was having the same problem and my MailScanner was marking every as DDos attack and all the server was deadslow . what i did to solve it 1. first of all check your ClamD (my clamd was outdated version) cleanly uninstall your clamd and then reinstall 2. i updated all the server now its been 15 days that my system running smothly

if your clamd is fine and latest i.e 0.99.2 then also check etc/Mailscanner/Mailscanner.conf

Maximum Processing Attempts = 0 make it 0 from 6 hope it will resolve the problem ............

[branko77]

My clamd is ok - ClamAV 0.99.2/22952/Fri Jan 27 08:52:36 2017

I just changed that Maximum Processing Attempts to 0 and started it again. For a whole week I had at least one crash within a 24 hours so now I need to wait to see if @ihsankhattak found a clue.

[branko77]

In /etc/clamd.conf from memory - search for "bind" and you'll probably see a note above saying it's best to bind it to localhost.

I did this as well. Found commented:

TCP address. By default we bind to INADDR_ANY, probably not wise. Enable the following to provide some degree of protection from the outside world. This option can be specified multiple times if you want to listen on multiple IPs. IPv6 is now supported. Default: no TCPAddr 127.0.0.1

and removed # in front of the TCPAddr 127.0.0.1 Of course I restarted all clam services afterwards.

[branko77]

ClamAV 0.99.2

On Jan 27, 2017 7:02 PM, "asuweb" notifications@github.com wrote:

@branko77 https://github.com/branko77 what version of ClamD are you running?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mailwatch/1.2.0/issues/430#issuecomment-275731200, or mute the thread https://github.com/notifications/unsubscribe-auth/AXbXsOK2taOX_0BnwtndkpYrwSn3kclCks5rWjEegaJpZM4Lq9fW .

[branko77]

Ok, I just had same f****g problem again. Second time today. I did that clam bind and changed maximum process attempts to zero but no luck...

[asuweb]

I'll hopefully get around to spinning up a centos 7 node next week and I'll let you know how I get on (plus I'll document it)

[asuweb]

Hasn't missed a beat on centos 6 though

[branko77]

I'll be more than thankfull for your effort. For now I switched off MailWatch. Just to much stress...

[stefaweb]

One more question. What is the DBD::mysql version on your system?

[branko77]

my DBD::mysql is version 4.041

[ihsankhattak]

Hi branko77 did you use clamav or clamd . i think in your setup you need clamd only and and in dependencies the clamav install .... which one command you use for clam installation ? also did you upgrade your clamav directly or first you did uninstall the clamav then reinstall ? ......
I am sure the problem is from clamav not in configuration ....

[asuweb]

@branko77 - are freshclam updates working for you? I'm in the process of building the centos 7 test server, and hit a snag with epel provided clamav. The mirror list is outdated and freshclam hangs. I wonder if that might be the issue you are facing?

Update: Removing and then re installing clamav seems to have cleared the issue. I let the mailscanner install script handle it initially.

[danitaz]

This has happened to me many times, but has not recurred since I added more memory to my VM about 2 weeks ago. Rebooting didn't even help me. I was also getting the DDoS attack messages during the time it was causing issues.

[branko77]

@ihsankhattak I installed clamav via yum. I have 2 services running that are related to clamav. They are: clamd and clam-freshclam. I am not sure anymore, but I think that I did fresh install. Maybe you are right about clamav, but I am not sure where to look for a problem...

[branko77]

@asuweb - I think that freshclam updates working just fine.

[root@******** /]# freshclam
ClamAV update process started at Sat Jan 28 09:06:32 2017
main.cvd is up to date (version: 57, sigs: 4218790, f-level: 60, builder: amishhammer)
daily.cld is up to date (version: 22957, sigs: 1475042, f-level: 63, builder: neo)
bytecode.cld is up to date (version: 290, sigs: 55, f-level: 63, builder: neo)

[ihsankhattak]

@branko77 i just want to understand your setup for example in my setup i am using the following programs 1.Mailwatch 2.MAilScanner 3.Postgrey 4.Postfix 5.ClamD not clamav clamav is running as dependency of clamd .

in clamd daemon is in used for scanning emails by using postfix configurations . Now if you have same kind of setup then first uninstall the clamav completly then reinstall clamd with command yum install clamd not clamav then freshclamd or freshclam then service clamd start then chkconfig --level 235 clamd on

i think it will resolve your problem and if it didn't resolve then try increase memory of your server also try to install rar and unrar in your server . in my openion it happens when clamd take too much time to scan the email and when time out then Mailscanner start marking it as Ddos attack...... hope it will resolve the problem

try then let us know about the status

[spec1re]

Sane problem here:

Jan 29 07:13:55 MS MailScanner[35733]: MailWatch: Logging message 112FC1C2AA6.A9ACA to SQL Jan 29 07:13:55 MS MailScanner[41299]: Could not use Custom Function code MailScanner::CustomConfig::InitMailWatchLogging, it could not be "eval"ed. Make sure the module is correct with perl -wc (Error: DBD::mysql::st execute failed: Incorrect string value: '\xF0\x9F\x92\x98\xC2\xA0...' for column 'subject' at row 1 at /usr/local/lib/MailScanner/MailScanner/CustomFunctions/MailWatch.pm line 173, line 1449. Jan 29 07:13:55 MS MailScanner[41299]: )

[branko77]

Yes that is warning that I get ocasionally on my cli...

On Jan 29, 2017 8:11 AM, "spec1re" notifications@github.com wrote:

Sane problem here:

Jan 29 07:13:55 MS MailScanner[35733]: MailWatch: Logging message 112FC1C2AA6.A9ACA to SQL Jan 29 07:13:55 MS MailScanner[41299]: Could not use Custom Function code MailScanner::CustomConfig::InitMailWatchLogging, it could not be "eval"ed. Make sure the module is correct with perl -wc (Error: DBD::mysql::st execute failed: Incorrect string value: '\xF0\x9F\x92\x98\xC2\xA0...' for column 'subject' at row 1 at /usr/local/lib/MailScanner/MailScanner/CustomFunctions/MailWatch.pm line 173, line 1449. Jan 29 07:13:55 MS MailScanner[41299]: )

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mailwatch/1.2.0/issues/430#issuecomment-275898235, or mute the thread https://github.com/notifications/unsubscribe-auth/AXbXsA-yoiKWsnwYwhY9RYVTPDakP78yks5rXDu0gaJpZM4Lq9fW .

[spec1re]

I can reproduced this bug 100%, sending this subject will crash MailWatch SQL logging:

Subject: =?utf-8?Q?=F0=9F=92=98=C2=A0Single=20or=20Taken=2C=20we=27re=20got=20you=20covered=20this=20V=2DDAY?=

It looks like an emoji is killing Mailwatch.

[asuweb]

@spec1re - excellent, that will give us something to focus on. Presumably you also got errors in the logs?

[spec1re]

Yes, the same line as I posted before.

[stefaweb]

For the ones using RC4, please run this new version of upgrade.php in #441

[spec1re]

Here my specs:

MailWatch Version 1.2.0 - RC4 MailScanner Version 4.85.2 ClamAV Version 0.99.2 SpamAssassin version 3.4.1 running on Perl version 5.24.1 PHP Version 5.5.30 MySQL Version 5.6.35-log

The database already converted to InnoDB and utf8mb4_unicode_ci.

[spec1re]

Testing connectivity to the database ................................. OK
Updating database schema: 
 - Convert database to utf8........................................... OK
 - Fix schema for username field in `users` table..................... OK
 - Fix schema for user field in `spamscores` table.................... OK
 - Fix schema for username field in `audit_log` table................. OK
 - Fix schema for password field in `users` table..................... OK
 - Fix schema for fullname field in `users` table..................... OK
 - Fix schema for username field in `user_filters` table.............. ERROR
Database error: Specified key was too long; max key length is 767 bytes - SQL = 'ALTER TABLE `user_filters` CHANGE `username` `username` VARCHAR( 255 ) NOT NULL DEFAULT '''

[spec1re]

VARCHAR( 191 ) did the trick ;)

Testing connectivity to the database ................................. OK
Updating database schema: 
 - Convert database to utf8........................................... OK
 - Fix schema for username field in `users` table..................... OK
 - Fix schema for user field in `spamscores` table.................... OK
 - Fix schema for username field in `audit_log` table................. OK
 - Fix schema for password field in `users` table..................... OK
 - Fix schema for fullname field in `users` table..................... OK
 - Fix schema for username field in `user_filters` table.............. OK
 - Fix schema for rule_desc field in `mcp_rules` table................ OK
 - Fix schema for msg_id field in `autorelease` table................. OK
 - Fix schema for uid field in `autorelease` table.................... OK
 - Convert database to utf8mb4........................................ OK
 - Convert table `audit_log` to utf8mb4............................... ALREADY CONVERTED
 - Convert table `autorelease` to utf8mb4............................. ALREADY CONVERTED
 - Convert table `blacklist` to utf8mb4............................... ALREADY CONVERTED
 - Convert table `inq` to utf8mb4..................................... ALREADY CONVERTED
 - Convert table `maillog` to utf8mb4................................. ALREADY CONVERTED
 - Convert table `mcp_rules` to utf8mb4............................... ALREADY CONVERTED
 - Convert table `mtalog` to utf8mb4.................................. ALREADY CONVERTED
 - Convert table `mtalog_ids` to utf8mb4.............................. ALREADY CONVERTED
 - Convert table `outq` to utf8mb4.................................... ALREADY CONVERTED
 - Convert table `saved_filters` to utf8mb4........................... ALREADY CONVERTED
 - Convert table `sa_rules` to utf8mb4................................ ALREADY CONVERTED
 - Convert table `spamscores` to utf8mb4.............................. ALREADY CONVERTED
 - Convert table `users` to utf8mb4................................... ALREADY CONVERTED
 - Convert table `user_filters` to utf8mb4............................ ALREADY CONVERTED
 - Convert table `whitelist` to utf8mb4............................... ALREADY CONVERTED
 - Drop `geoip_country` table......................................... ALREADY DROPPED

Checking MailScanner.conf settings: 
 - QuarantineWholeMessage ............................................ OK
 - QuarantineWholeMessagesAsQueueFiles ............................... OK
 - DetailedSpamReport ................................................ OK
 - IncludeScoresInSpamAssassinReport ................................. OK
 - SpamActions ....................................................... OK
 - HighScoringSpamActions ............................................ OK
 - AlwaysLookedUpLast ................................................ OK

[stefaweb]

Doesn't understand, username for user_filters is not a PRIMARY KEY!

CREATE TABLE IF NOT EXISTS `user_filters` (
  `username` varchar(255) COLLATE utf8_unicode_ci NOT NULL DEFAULT '',
  `filter` mediumtext COLLATE utf8_unicode_ci,
  `verify_key` varchar(32) COLLATE utf8_unicode_ci NOT NULL DEFAULT '',
  `active` enum('N','Y') COLLATE utf8_unicode_ci DEFAULT 'N',
  KEY `user_filters_username_idx` (`username`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

I patched upgraded.php for this.

[stefaweb]

Tried the same and no problem with mariadb 10.

Subject: =?utf-8?Q?=F0=9F=92=98=C2=A0Single=20or=20Taken=2C=20we=27re=20got=20you=20covered=20this=20V=2DDAY?=

[asuweb]

Subject: =?utf-8?Q?=F0=9F=92=98=C2=A0Single=20or=20Taken=2C=20we=27re=20got=20you=20covered=20this=20V=2DDAY?=

Tested on mine with no issues.

MailWatch Version: 1.2.0 - RC3

MailScanner Version: 5.0.4 SpamAssassin Version: 3.3.1 PHP Version: 5.6.29 MySQL Version: 5.1.73

[spec1re]

This is how I test it:

[asuweb]

I tested it by sending a message through the system with that as the subject - didn't complain

[stefaweb]

Here the detail message in MW.

emoji is not well printed on screen for the field Subject: but well printed in the headers!

Not well printed on list in status.php too.

[stefaweb]

I tested using this in the subject:

=?utf-8?Q?=F0=9F=92=98=C2=A0Single=20or=20Taken=2C=20we=27re=20got=20you=20covered=20this=20V=2DDAY?=

Entered directly like this (not clear text).

And now mailq.php crash all one minute (cron).

Jan 29 11:03:01 xxxx php: PHP Notice: Undefined index: subject in /usr/local/bin/mailq.php on line 266

In the database:

[stefaweb]

Shit. I can't delete the record from table maillog.

phpmyadmin say:

Current selection does not contain a unique column. Grid edit, checkbox, Edit, Copy and Delete features are not available.

[asuweb]

try it from the mysql command line

[spec1re]

Did a fresh DB create.sql and upgrade.php but still crashing, here the dbitrace.log

 DBI 1.636-ithread default trace level set to 0x0/2 (pid 7403 pi 802019e00) at MailWatch.pm line 33 via Config.pm line 760
    -> DBI->connect(DBI:mysql:database=mailscanner;host=localhost, mailwatch, ****, HASH(0x808262318))
    -> DBI->install_driver(mysql) for freebsd perl=5.024001 pid=7403 ruid=125 euid=125
       install_driver: DBD::mysql version 4.041 loaded from /usr/local/lib/perl5/site_perl/mach/5.24/DBD/mysql.pm
///cut
sqlite trace: Removing statement from list: 80e78e688 at dbdimp.c line 1268
!   <- DESTROY= ( undef ) [1 items] during global destruction
!   -> DESTROY for DBD::SQLite::db (DBI::db=HASH(0x80bbf5d68)~INNER) thr#802019e00
         DESTROY DBI::db=HASH(0x80bbf5d68) skipped due to InactiveDestroy
!   <- DESTROY= ( undef ) [1 items] during global destruction
!   -> DESTROY for DBD::SQLite::db (DBI::db=HASH(0x807fb1318)~INNER) thr#802019e00
         DESTROY DBI::db=HASH(0x807fb1318) skipped due to InactiveDestroy
!   <- DESTROY= ( undef ) [1 items] during global destruction
!   -> DESTROY in DBD::_::common for DBD::SQLite::dr (DBI::dr=HASH(0x80b362d80)~INNER) thr#802019e00
!   <- DESTROY= ( undef ) [1 items] during global destruction
!   -> DESTROY in DBD::_::common for DBD::mysql::dr (DBI::dr=HASH(0x807ed9a08)~INNER) thr#802019e00
!   <- DESTROY= ( undef ) [1 items] during global destruction
    -> execute for DBD::SQLite::st (DBI::st=HASH(0x80f077618)~0x80f05d678 '907191C2B04.A8083') thr#802019e00
    <- execute= ( 1 ) [1 items] at MessageBatch.pm line 1314
    -> ping for DBD::mysql::db (DBI::db=HASH(0x80826e540)~0x8031a7f18) thr#802019e00
    <- ping= ( 1 ) [1 items] at MailWatch.pm line 136
    -> execute for DBD::mysql::st (DBI::st=HASH(0x80826de40)~0x8082622e8 '2017-01-29 11:00:55' '907191C2B04.A8083' 1465 'noreply@test.de' 'test.de' '123@123.de' '123.de' "💘 Single or Taken, we're got you covered this V-DAY" '213.252.49.7' '' 1 0 1 undef 0 0 3.174 'spam, SpamAssassin (nicht zwischen gespeichert, Wertung=3.174, benoetigt 3, BAYES_50 0.80, FSL_HELO_NON_FQDN_1 0.00, HTML_MESSAGE 0.00, KAM_LAZY_DOMAIN_SECURITY 0.10, RDNS_NONE 0.10, TVD_RCVD_SINGLE 2.17)' 0 0 0 '' undef undef undef undef undef undef undef 'zonk.jf.de' '2017-01-29' '11:00:55' 'Received: from BLACKHAWK (unknown [213.252.49.7])
    by zonk.janus-it.com (Postfix) with ESMTP id 907191C2B04
    for <123@123.de>; Sun, 29 Jan 2017 11:00:53 +0100 (CET)
To: 123@123.de
From:   noreply@test.de
Subject:
 =?utf-8?Q?=F0=9F=92=98=C2=A0Single=20or=20Taken=2C=20we=27re=20got=20you=20covered=20this=20V=2DDAY?=
Date: Sun, 29 Jan 2017 11:01:01 +0100
Message-Id: <f76a34675f7960657be0defbf77efc5a@test.de>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="boundaryLr+IjQ=="' 1) thr#802019e00
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
   Called: dbd_bind_ph
 -> dbd_st_execute for 80826eed0
    >- dbd_st_free_result_sets
    <- dbd_st_free_result_sets RC -1
    <- dbd_st_free_result_sets
mysql_st_internal_execute MYSQL_VERSION_ID 50635
>parse_params statement INSERT INTO maillog (timestamp, id, size, from_address, from_domain, to_address, to_domain, subject, clientip, archive, isspam, ishighspam, issaspam, isrblspam, spamwhitelisted, spamblacklisted, sascore, spamreport, virusinfected, nameinfected, otherinfected, report, ismcp, ishighmcp, issamcp, mcpwhitelisted, mcpblacklisted, mcpsascore, mcpreport, hostname, date, time, headers, quarantined) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
Binding parameters: INSERT INTO maillog (timestamp, id, size, from_address, from_domain, to_address, to_domain, subject, clientip, archive, isspam, ishighspam, issaspam, isrblspam, spamwhitelisted, spamblacklisted, sascore, spamreport, virusinfected, nameinfected, otherinfected, report, ismcp, ishighmcp, issamcp, mcpwhitelisted, mcpblacklisted, mcpsascore, mcpreport, hostname, date, time, headers, quarantined) VALUES ('2017-01-29 11:00:55','907191C2B04.A8083','1465','noreply@test.de','test.de','123@123.de','123.de','💘 Single or Taken, we\'re got you covered this V-DAY','213.252.49.7','','1','0','1',NULL,'0','0','3.174','spam, SpamAssassin (nicht zwischen gespeichert, Wertung=3.174, benoetigt 3, BAYES_50 0.80, FSL_HELO_NON_FQDN_1 0.00, HTML_MESSAGE 0.00, KAM_LAZY_DOMAIN_SECURITY 0.10, RDNS_NONE 0.10, TVD_RCVD_SINGLE 2.17)','0','0','0','',NULL,NULL,NULL,NULL,NULL,NULL,NULL,'zonk.jf.de','2017-01-29','11:00:55','Received: from BLACKHAWK (unknown [213.252.49.7])\n   by zonk.janus-it.com (Postfix) with ESMTP id 907191C2B04\n  for <123@123.de>; Sun, 29 Jan 2017 11:00:53 +0100 (CET)\nTo: 123@123.de\nFrom:  noreply@test.de\nSubject:\n =?utf-8?Q?=F0=9F=92=98=C2=A0Single=20or=20Taken=2C=20we=27re=20got=20you=20covered=20this=20V=2DDAY?=\nDate: Sun, 29 Jan 2017 11:01:01 +0100\nMessage-Id: <f76a34675f7960657be0defbf77efc5a@test.de>\nMIME-Version: 1.0\nContent-Type: multipart/alternative; boundary=\"boundaryLr+IjQ==\"','1')
        --> do_error
Incorrect string value: '\xF0\x9F\x92\x98\xC2\xA0...' for column 'subject' at row 1 error 1366 recorded: Incorrect string value: '\xF0\x9F\x92\x98\xC2\xA0...' for column 'subject' at row 1
        <-- do_error
IGNORING ERROR errno 1366
 <- dbd_st_execute returning imp_sth->row_num 18446744073709551614
    !! ERROR: 1366 'Incorrect string value: '\xF0\x9F\x92\x98\xC2\xA0...' for column 'subject' at row 1' (err#0)
    <- execute= ( undef ) [1 items] at MailWatch.pm line 173
    -- DBI::END ($@: , $!: )
    -> disconnect_all for DBD::mysql::dr (DBI::dr=HASH(0x80826daf8)~0x80826dd20) thr#802019e00
    <- disconnect_all= ( ) [0 items] (not implemented) at DBI.pm line 763
!   -> DESTROY for DBD::mysql::db (DBI::db=HASH(0x8031a7f18)~INNER) thr#802019e00
imp_dbh->pmysql: 805be3100
       ERROR: 1366 'Incorrect string value: '\xF0\x9F\x92\x98\xC2\xA0...' for column 'subject' at row 1' (err#0)
!   <- DESTROY= ( undef ) [1 items] during global destruction
!   -> DESTROY for DBD::mysql::st (DBI::st=HASH(0x8082622e8)~INNER) thr#802019e00
    Freeing 34 parameters, bind 0 fbind 0
       ERROR: 1366 'Incorrect string value: '\xF0\x9F\x92\x98\xC2\xA0...' for column 'subject' at row 1' (err#0)
!   <- DESTROY= ( undef ) [1 items] during global destruction
!   -> DESTROY in DBD::_::common for DBD::mysql::dr (DBI::dr=HASH(0x80826dd20)~INNER) thr#802019e00
!   <- DESTROY= ( undef ) [1 items] during global destruction
    -> selectrow_array for DBD::SQLite::db (DBI::db=HASH(0x80bbfe3a8)~0x80bbfe2e8 DBI::st=HASH(0x80f5196a8) undef '29A141C2B16.A9B04') thr#802019e00
    <- selectrow_array= ( ) [0 items] at Postfix.pm line 1803
    -> execute for DBD::SQLite::st (DBI::st=HASH(0x80edd6cf0)~0x80f519798 '29A141C2B16.A9B04' 1 1485684241) thr#802019e00
    <- execute= ( 1 ) [1 items] at Postfix.pm line 1844
    -- DBI::END ($@: , $!: )
    -> disconnect_all for DBD::SQLite::dr (DBI::dr=HASH(0x80b361d50)~0x80b361dc8) thr#802019e00
    <- disconnect_all= ( '' ) [1 items] at DBI.pm line 763
    -> disconnect_all for DBD::mysql::dr (DBI::dr=HASH(0x807ed9708)~0x807ed9a50) thr#802019e00
    <- disconnect_all= ( ) [0 items] (not implemented) at DBI.pm line 763
!   -> DESTROY for DBD::SQLite::st (DBI::st=HASH(0x80f519798)~INNER) thr#802019e00

[stefaweb]

I deleted the record manually but mailq.php continue to crash. I rebooted the dev server and now its gone!

The maillog schema need to be updated and stronger filtering added to maillog input.

[asuweb]

It works on mine without crashing

[root@mx1 alanU]# mail -s "=?utf-8?Q?=F0=9F=92=98=C2=A0Single=20or=20Taken=2C=20we=27re=20got=20you=20covered=20this=20V=2DDAY?=" delivery-test@asuweb.co.uk < /dev/null

Jan 29 10:25:49 mx1 MailScanner[22982]: New Batch: Scanning 1 messages, 1246 bytes
Jan 29 10:25:49 mx1 MailScanner[22982]: Virus and Content Scanning: Starting
Jan 29 10:26:00 mx1 MailScanner[22982]: Virus Scanning completed at 117 bytes per second
Jan 29 10:26:00 mx1 MailScanner[22982]: Spam Checks: Starting
Jan 29 10:26:00 mx1 MailScanner[22982]: Whitelist refresh time reached
Jan 29 10:26:00 mx1 MailScanner[22982]: Starting up SQL Whitelist
Jan 29 10:26:00 mx1 MailScanner[22982]: Read 77 whitelist entries
Jan 29 10:26:00 mx1 MailScanner[22982]: Blacklist refresh time reached
Jan 29 10:26:00 mx1 MailScanner[22982]: Starting up SQL Blacklist
Jan 29 10:26:00 mx1 MailScanner[22982]: Read 4 blacklist entries
Jan 29 10:26:01 mx1 MailScanner[22982]: Spam Checks completed at 729 bytes per second
Jan 29 10:26:01 mx1 MailScanner[22982]: Requeue: 53D21632E1.AB9CF to 76C476320D
Jan 29 10:26:01 mx1 MailScanner[22982]: Uninfected: Delivered 1 messages
Jan 29 10:26:01 mx1 MailScanner[22982]: Virus Processing completed at 251158 bytes per second
Jan 29 10:26:01 mx1 MailScanner[22982]: Deleted 1 messages from processing-database
Jan 29 10:26:01 mx1 MailScanner[22982]: Batch completed at 101 bytes per second (1246 / 12)
Jan 29 10:26:01 mx1 MailScanner[22982]: Batch (1 message) processed in 12.30 seconds
Jan 29 10:26:01 mx1 MailScanner[22982]: Logging message 53D21632E1.AB9CF to SQL
Jan 29 10:26:01 mx1 MailScanner[22982]: "Always Looked Up Last" took 0.00 seconds
[root@mx1 Git]# grep '53D21632E1.AB9CF' /var/log/maillog
Jan 29 10:26:01 mx1 MailScanner[22982]: Requeue: 53D21632E1.AB9CF to 76C476320D
Jan 29 10:26:01 mx1 MailScanner[22982]: Logging message 53D21632E1.AB9CF to SQL
Jan 29 10:26:01 mx1 MailScanner[7401]: 53D21632E1.AB9CF: Logged to MailWatch SQL
[root@mx1 Git]# grep '76C476320D' /var/log/maillog
Jan 29 10:26:01 mx1 MailScanner[22982]: Requeue: 53D21632E1.AB9CF to 76C476320D
Jan 29 10:26:01 mx1 postfix/qmgr[2351]: 76C476320D: from=<root@asuantispam.com>, size=558, nrcpt=1 (queue active)
Jan 29 10:26:02 mx1 postfix/smtp[8410]: 76C476320D: to=<delivery-test@asuweb.co.uk>, relay=149.202.177.126[149.202.177.126]:25, delay=13, delays=13/0.01/0.18/0.03, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 25EDFC2451)
Jan 29 10:26:02 mx1 postfix/qmgr[2351]: 76C476320D: removed

[stefaweb]

I directly sent the crashing mail from my Mac using Outlook not using the MW server.

The mail is well received later on the Mac unless MW crash.

Its like the story of the crashing SMS on the iPhone... ;)

[stefaweb]

Line 266 in mailq.php

    ('" . safe_value($msgid) . "','" .
                    safe_value($msginfo['cdate']) . "','" .
                    safe_value($msginfo['ctime']) . "','" .
                    safe_value($msginfo['sender']) . "','" .
                    safe_value(@implode(',', $msginfo['rcpts'])) . "','" .
                    safe_value($msginfo['subject']) . "','" .
                    safe_value($msginfo['message']) . "','" .
                    safe_value($msginfo['size']) . "','" .
                    safe_value($msginfo['priority']) . "','" .
                    safe_value($msginfo['attempts']) . "','" .
                    safe_value($msginfo['lastattempttime']) . "','" .
                    safe_value($sys_hostname) . "')";
                dbquery($sql);
            }
        }
    }

And function safe_value:

function safe_value($value)
{
    $link = dbconn();
    if (function_exists('get_magic_quotes_gpc') && get_magic_quotes_gpc()) {
        $value = stripslashes($value);
    }
    $value = $link->real_escape_string($value);

    return $value;
}

[asuweb]

I've tried it in every different way, and I can't make mine crash at all.

[stefaweb]

You're using RC3, we are using RC4.

[asuweb]

Ahh yes, on that server (the live one) I am indeed using RC3 - that should tell you where the bug is then :)

[asuweb]

Comparing the diff between the 2 branches, I can't see anything relevant which has changed

[asuweb]

What Mailscanner version are you using?

[branko77]

5.0.3

On Jan 29, 2017 11:48, "asuweb" notifications@github.com wrote:

What Mailscanner version are you using?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mailwatch/1.2.0/issues/430#issuecomment-275906274, or mute the thread https://github.com/notifications/unsubscribe-auth/AXbXsEjyozIl15SjHRhpAy5z9rmE1Og7ks5rXG5mgaJpZM4Lq9fW .

[stefaweb]

Here my specs (Debian 8):

MailWatch Version 1.2.0 - RC4 (and some patch from the current works) MailScanner Version 5.0.3-7 ClamAV Version 0.99.2+dfsg-0+deb8u2 SpamAssassin version 3.4.0-6 Perl version 5.20.2-3+deb8u6 PHP Version 5.6.29+dfsg-0+deb8u1 libdbi-perl 1.631-3+b1 libdbd-mysql-perl 4.028-2+deb8u2 mariadb Version 10.0.29+maria-1~jessie

All regular, no backport or manual CPAN upgrade.

[asuweb]

OK,

Let me spin up 2 dev servers, one Debian / 1 Centos.

I'll use the latest mailscanner package (5.0.3-7) and the mailwatch master branch.

Stock packages for everything else.

That way we've got a clean environment from which to look at this