branko77 commented 7 years ago

Today my MailWatch just stopped logging in to MySQL. Last Message that has been logged was few hours ago. My postfix is working fine and I can send and receive messages. I received few of messages with this text: "MailScanner was attacked by a Denial Of Service attack, and has therefore deleted this part of the message. Please contact your e-mail providers for more information if you need it, giving them the whole of this report. Attack in: /var/spool/MailScanner/incoming/36423/4335498CC.AB426/nmsg-36423-81.html " After reboot of my server my NAGIOS is reporting that Clamd is critical with error "connect to address 127.0.0.1 and port 3310: Connection refused" When I check MySQL I can see that last message was logged few hours ago and that is not logging. maillog reports that MailWatch: Logging message 5D18798D8.A7796 to SQL but no logged message i maillog. I didn't made any changes what so ever...

spec1re commented 7 years ago

When I compose a mail on my iPhone, with some random emojis in the subject and send it to the Postfix/MS/MW server = instant crash.

asuweb commented 7 years ago

Well, I've just finished building the 2 dev servers, so lets try it now.

@spec1re - if you could send the email that's crashing yours to these 2 addresses, we'll see if it crashes the 2 newly built servers:

test@debian.dev.asuantispam.com test@centos.dev.asuantispam.com

spec1re commented 7 years ago

The mails are send.

stefaweb commented 7 years ago

Some related problems.

Here: #238

http://stackoverflow.com/questions/10957238/incorrect-string-value-when-trying-to-insert-utf-8-into-mysql-via-jdbc

http://www.perlmonks.org/?node_id=745538

http://dba.stackexchange.com/questions/89355/unable-to-insert-utf8mb4-characters-in-mysql-5-6

Changing default setup in /etc/mysql/my.conf is not a valuable solution.

asuweb commented 7 years ago

Ok, well on both boxes they worked like a charm and the emails were processed normally.

Log outputs follow:

Debian:

Jan 29 13:51:48 template MailScanner[27667]: New Batch: Scanning 1 messages, 3654 bytes
Jan 29 13:51:48 template MailScanner[27667]: Virus and Content Scanning: Starting
Jan 29 13:52:00 template MailScanner[27667]: Virus Scanning completed at 308 bytes per second
Jan 29 13:52:00 template MailScanner[27667]: Spam Checks: Starting
Jan 29 13:52:00 template MailScanner[27667]: Spam Checks completed at 38850 bytes per second
Jan 29 13:52:00 template MailScanner[27667]: Requeue: 76C4B4078C.A712E to DCD0D407C8
Jan 29 13:52:00 template MailScanner[27667]: Uninfected: Delivered 1 messages
Jan 29 13:52:00 template MailScanner[27667]: Virus Processing completed at 751531 bytes per second
Jan 29 13:52:00 template MailScanner[27667]: Deleted 1 messages from processing-database
Jan 29 13:52:00 template MailScanner[27667]: Batch completed at 306 bytes per second (3654 / 11)
Jan 29 13:52:00 template MailScanner[27667]: Batch (1 message) processed in 11.94 seconds
Jan 29 13:52:00 template MailScanner[27667]: MailWatch: Logging message 76C4B4078C.A712E to SQL
Jan 29 13:52:00 template MailScanner[27667]: "Always Looked Up Last" took 0.00 seconds
Jan 29 13:52:42 template MailScanner[27667]: New Batch: Scanning 1 messages, 3226 bytes
Jan 29 13:52:42 template MailScanner[27667]: Virus and Content Scanning: Starting
Jan 29 13:52:54 template MailScanner[27667]: Virus Scanning completed at 274 bytes per second
Jan 29 13:52:54 template MailScanner[27667]: Spam Checks: Starting
Jan 29 13:52:54 template MailScanner[27667]: Spam Checks completed at 43979 bytes per second
Jan 29 13:52:54 template MailScanner[27667]: Requeue: 52BE84078C.AE91A to DE559407C9
Jan 29 13:52:54 template MailScanner[27667]: Uninfected: Delivered 1 messages
Jan 29 13:52:54 template MailScanner[27667]: Virus Processing completed at 846523 bytes per second
Jan 29 13:52:54 template MailScanner[27667]: Deleted 1 messages from processing-database
Jan 29 13:52:54 template MailScanner[27667]: Batch completed at 272 bytes per second (3226 / 11)
Jan 29 13:52:54 template MailScanner[27667]: Batch (1 message) processed in 11.84 seconds
Jan 29 13:52:54 template MailScanner[27667]: MailWatch: Logging message 52BE84078C.AE91A to SQL
Jan 29 13:52:54 template MailScanner[27667]: "Always Looked Up Last" took 0.00 seconds

Jan 29 13:52:41 template postfix/smtpd[28231]: 52BE84078C: client=st14p37im-asmtp001.me.com[17.41.209.30]
Jan 29 13:52:41 template postfix/cleanup[28235]: 52BE84078C: hold: header Received: from st14p37im-asmtp001.me.com (st14p37im-asmtp001.me.com [17.41.209.30])??(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))??(No client certificate requested)??by devde from st14p37im-asmtp001.me.com[17.41.209.30]; from=<redacted@icloud.com> to=<test@debian.dev.asuantispam.com> proto=ESMTP helo=<st14p37im-asmtp001.me.com>
Jan 29 13:52:41 template postfix/cleanup[28235]: 52BE84078C: message-id=<D7B0A79A-3481-4332-8336-88583654FBE1@icloud.com>
Jan 29 13:52:54 template MailScanner[27667]: Requeue: 52BE84078C.AE91A to DE559407C9
Jan 29 13:52:54 template MailScanner[27667]: MailWatch: Logging message 52BE84078C.AE91A to SQL
Jan 29 13:52:54 template MailScanner[27667]: Requeue: 52BE84078C.AE91A to DE559407C9
Jan 29 13:52:54 template postfix/qmgr[27970]: DE559407C9: from=<redacted@icloud.com>, size=2396, nrcpt=1 (queue active)
Jan 29 13:52:54 template postfix/smtp[28261]: DE559407C9: to=<test@debian.dev.asuantispam.com>, relay=149.202.177.126[149.202.177.126]:25, delay=13, delays=13/0/0.09/0.03, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 0794CC2B6A)
Jan 29 13:52:54 template postfix/qmgr[27970]: DE559407C9: removed

Centos

Jan 29 14:53:49 devcentos postfix/smtpd[32349]: 16B0A207E5: client=st14p37im-asmtp001.me.com[17.41.209.30]
Jan 29 14:53:49 devcentos postfix/cleanup[32352]: 16B0A207E5: hold: header Received: from st14p37im-asmtp001.me.com (st14p37im-asmtp001.me.com [17.41.209.30])??(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))??(No client certificate requested)??by devce from st14p37im-asmtp001.me.com[17.41.209.30]; from=<redacted@icloud.com> to=<test@centos.dev.asuantispam.com> proto=ESMTP helo=<st14p37im-asmtp001.me.com>
Jan 29 14:53:49 devcentos postfix/cleanup[32352]: 16B0A207E5: message-id=<6DF7A03A-0F09-47E9-ADC7-AB24EB18EB78@icloud.com>
Jan 29 14:54:05 devcentos MailScanner[30409]: Requeue: 16B0A207E5.AE551 to 587EC20815
Jan 29 14:54:05 devcentos MailScanner[30409]: MailWatch: Logging message 16B0A207E5.AE551 to SQL

Jan 29 14:54:05 devcentos MailScanner[30409]: Requeue: 16B0A207E5.AE551 to 587EC20815
Jan 29 14:54:05 devcentos postfix/qmgr[31057]: 587EC20815: from=<redacted@icloud.com>, size=2361, nrcpt=1 (queue active)
Jan 29 14:54:06 devcentos postfix/smtp[32381]: 587EC20815: to=<test@centos.dev.asuantispam.com>, relay=149.202.177.126[149.202.177.126]:25, delay=17, delays=17/0.01/0.09/0.03, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as B469DC2B6A)
Jan 29 14:54:06 devcentos postfix/qmgr[31057]: 587EC20815: removed

Jan 29 14:53:51 devcentos MailScanner[30409]: New Batch: Scanning 1 messages, 3190 bytes
Jan 29 14:53:51 devcentos MailScanner[30409]: Virus and Content Scanning: Starting
Jan 29 14:54:04 devcentos MailScanner[30409]: Virus Scanning completed at 238 bytes per second
Jan 29 14:54:04 devcentos MailScanner[30409]: Spam Checks: Starting
Jan 29 14:54:05 devcentos MailScanner[30409]: Spam Checks completed at 3098 bytes per second
Jan 29 14:54:05 devcentos MailScanner[30409]: Requeue: 16B0A207E5.AE551 to 587EC20815
Jan 29 14:54:05 devcentos MailScanner[30409]: Uninfected: Delivered 1 messages
Jan 29 14:54:05 devcentos MailScanner[30409]: Deleted 1 messages from processing-database
Jan 29 14:54:05 devcentos MailScanner[30409]: Batch completed at 221 bytes per second (3190 / 14)
Jan 29 14:54:05 devcentos MailScanner[30409]: Batch (1 message) processed in 14.40 seconds
Jan 29 14:54:05 devcentos MailScanner[30409]: MailWatch: Logging message 16B0A207E5.AE551 to SQL
Jan 29 14:54:05 devcentos MailScanner[30409]: "Always Looked Up Last" took 0.00 seconds

Centos

Debian

spec1re commented 7 years ago

Yippee! Not crashing anymore:

unbenannt123

Fix was to edit my.cnf and add:

[client]
default-character-set = utf8mb4

[mysql]
default-character-set = utf8mb4

[mysqld]
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci

The default my.cnf shipped via pkg in FreeBSD is missing those values.

Thank you all so much! :)

stefaweb commented 7 years ago

And collation?

spec1re commented 7 years ago

Do I still miss a something? Here my full my.cnf:

# $FreeBSD$

[client]
port                            = 3306
socket                          = /tmp/mysql.sock
default-character-set = utf8mb4

[mysql]
prompt                          = \u@\h [\d]>\_
no_auto_rehash
default-character-set = utf8mb4

[mysqld]
user                            = mysql
port                            = 3306
socket                          = /tmp/mysql.sock
bind-address                    = 127.0.0.1
basedir                         = /usr/local
datadir                         = /var/db/mysql
tmpdir                          = /var/db/mysql_tmpdir
slave-load-tmpdir               = /var/db/mysql_tmpdir
secure-file-priv                = /var/db/mysql_secure
log-bin                         = mysql-bin
log-output                      = TABLE
master-info-repository          = TABLE
relay-log-info-repository       = TABLE
relay-log-recovery              = 1
slow-query-log                  = 1
server-id                       = 1
sync_binlog                     = 1
sync_relay_log                  = 1
binlog_cache_size               = 16M
expire_logs_days                = 30
log-slave-updates               = 1
enforce-gtid-consistency        = 1
gtid-mode                       = ON
safe-user-create                = 1
lower_case_table_names          = 1
explicit-defaults-for-timestamp = 1
myisam-recover-options          = BACKUP,FORCE
open_files_limit                = 32768
table_open_cache                = 16303
table_definition_cache          = 8192
net_retry_count                 = 16384
key_buffer_size                 = 256M
max_allowed_packet              = 64M
query_cache_type                = 0
query_cache_size                = 0
long_query_time                 = 0.5
innodb_buffer_pool_size         = 1G
innodb_data_home_dir            = /var/db/mysql
innodb_log_group_home_dir       = /var/db/mysql
innodb_data_file_path           = ibdata1:128M:autoextend
innodb_flush_method             = O_DIRECT
innodb_log_file_size            = 256M
innodb_log_buffer_size          = 16M
innodb_write_io_threads         = 8
innodb_read_io_threads          = 8
innodb_autoinc_lock_mode        = 2
skip-symbolic-links
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci

[mysqldump]
max_allowed_packet              = 256M
quote_names
quick

branko77 commented 7 years ago

I'll try in the morning with my.cnf and let you know

On Jan 29, 2017 15:23, "Stéphane" notifications@github.com wrote:

And collation?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mailwatch/1.2.0/issues/430#issuecomment-275916931, or mute the thread https://github.com/notifications/unsubscribe-auth/AXbXsKGzPTvuh8syz2Omibl18v9CXisyks5rXKDmgaJpZM4Lq9fW .

stefaweb commented 7 years ago

For reference, Debian 8 mariadb default.

/etc/mysql/conf.d/mariadb.cnf

# MariaDB-specific config file.
# Read by /etc/mysql/my.cnf

[client]
# Default is Latin1, if you need UTF-8 set this (also in server section)
#default-character-set = utf8

[mysqld]
#
# * Character sets
#
# Default is Latin1, if you need UTF-8 set all this (also in client section)
#
#character-set-server  = utf8
#collation-server      = utf8_general_ci
#character_set_server   = utf8
#collation_server       = utf8_general_ci

in /etc/mysql/my.cnf

# MariaDB database server configuration file.
#
# You can copy this file to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html

# This will be passed to all mysql clients
# It has been reported that passwords should be enclosed with ticks/quotes
# escpecially if they contain "#" chars...
# Remember to edit /etc/mysql/debian.cnf when changing the socket location.
[client]
port        = 3306
socket      = /var/run/mysqld/mysqld.sock

# Here is entries for some specific programs
# The following values assume you have at least 32M ram

# This was formally known as [safe_mysqld]. Both versions are currently parsed.
[mysqld_safe]
socket      = /var/run/mysqld/mysqld.sock
nice        = 0

[mysqld]
#
# * Basic Settings
#
user        = mysql
pid-file    = /var/run/mysqld/mysqld.pid
socket      = /var/run/mysqld/mysqld.sock
port        = 3306
basedir     = /usr
datadir     = /var/lib/mysql
tmpdir      = /tmp
lc_messages_dir = /usr/share/mysql
lc_messages = en_US
skip-external-locking
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address        = 127.0.0.1
#
# * Fine Tuning
#
max_connections     = 100
connect_timeout     = 5
wait_timeout        = 600
max_allowed_packet  = 16M
thread_cache_size       = 128
sort_buffer_size    = 4M
bulk_insert_buffer_size = 16M
tmp_table_size      = 32M
max_heap_table_size = 32M
#
# * MyISAM
#
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched. On error, make copy and try a repair.
myisam_recover          = BACKUP
key_buffer_size     = 128M
#open-files-limit   = 2000
table_open_cache    = 400
myisam_sort_buffer_size = 512M
concurrent_insert   = 2
read_buffer_size    = 2M
read_rnd_buffer_size    = 1M
#
# * Query Cache Configuration
#
# Cache only tiny result sets, so we can fit more in the query cache.
query_cache_limit       = 128K
query_cache_size        = 64M
# for more write intensive setups, set to DEMAND or OFF
#query_cache_type       = DEMAND
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file        = /var/log/mysql/mysql.log
#general_log             = 1
#
# Error logging goes to syslog due to /etc/mysql/conf.d/mysqld_safe_syslog.cnf.
#
# we do want to know about network errors and such
log_warnings        = 2
#
# Enable the slow query log to see queries with especially long duration
#slow_query_log[={0|1}]
slow_query_log_file = /var/log/mysql/mariadb-slow.log
long_query_time = 10
#log_slow_rate_limit    = 1000
log_slow_verbosity  = query_plan

#log-queries-not-using-indexes
#log_slow_admin_statements
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
#       other settings you may need to change.
#server-id      = 1
#report_host        = master1
#auto_increment_increment = 2
#auto_increment_offset  = 1
log_bin         = /var/log/mysql/mariadb-bin
log_bin_index       = /var/log/mysql/mariadb-bin.index
# not fab for performance, but safer
#sync_binlog        = 1
expire_logs_days    = 10
max_binlog_size         = 100M
# slaves
#relay_log      = /var/log/mysql/relay-bin
#relay_log_index    = /var/log/mysql/relay-bin.index
#relay_log_info_file    = /var/log/mysql/relay-bin.info
#log_slave_updates
#read_only
#
# If applications support it, this stricter sql_mode prevents some
# mistakes like inserting invalid dates etc.
#sql_mode       = NO_ENGINE_SUBSTITUTION,TRADITIONAL
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
default_storage_engine  = InnoDB
# you can't just change log file size, requires special procedure
#innodb_log_file_size   = 50M
innodb_buffer_pool_size = 256M
innodb_log_buffer_size  = 8M
innodb_file_per_table   = 1
innodb_open_files   = 400
innodb_io_capacity  = 400
innodb_flush_method = O_DIRECT
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem

[mysqldump]
quick
quote-names
max_allowed_packet  = 16M

[mysql]
#no-auto-rehash # faster start of mysql but no tab completition

[isamchk]
key_buffer      = 16M

#
# * IMPORTANT: Additional settings that can override those from this file!
#   The files must end with '.cnf', otherwise they'll be ignored.
#
!includedir /etc/mysql/conf.d/

spec1re commented 7 years ago

Here my table layout:

unbenannt1234

Excellent!

unbenannt12345

stefaweb commented 7 years ago

Changed settings. No longer crash.

Does changing this in mysql/mariadb configs can be a problem if others application using mysql are running on the server?

Can we find a fix without editing mysql configs?

asuweb commented 7 years ago

Glad you've found the solution - definitely not a mailwatch bug

spec1re commented 7 years ago

MySQL is dedicated for MailWatch, so for now it should not be a problem.

stefaweb commented 7 years ago

Seems we can code this directly in MW. No need to upgrade configs.

Solution two (don't need to restart MySQL):

1.change the table and column encoding to utf8mb4

2.specifying characterEncoding=UTF-8 in the jdbc connector, cause the jdbc connector doesn't suport utf8mb4.

3.write your sql statment like this (need to add allowMultiQueries=true to jdbc connector):

'SET NAMES utf8mb4;INSERT INTO Mytable ...';

this will make sure each connection to the server, character_set_client, character_set_connection, character_set_results are utf8mb4.

Point 2 is may be no longer accurate as the comment is 4 years old.

spec1re commented 7 years ago

That would be even better! :)

stefaweb commented 7 years ago

Line 266 in MailWatch.pm:

    # Place all data into %msg
    my %msg;
    $msg{timestamp} = $timestamp;
    $msg{id} = $message->{id};
    $msg{size} = $message->{size};
    $msg{from} = $message->{from};
    $msg{from_domain} = $message->{fromdomain};
    $msg{to} = join(",", @{$message->{to}});
    $msg{to_domain} = $todomain;
    $msg{subject} = fix_latin($message->{utf8subject});
    $msg{clientip} = $clientip;
    $msg{archiveplaces} = join(",", @{$message->{archiveplaces}});
    $msg{isspam} = $message->{isspam};
    $msg{ishigh} = $message->{ishigh};
    $msg{issaspam} = $message->{issaspam};
    $msg{isrblspam} = $message->{isrblspam};
    $msg{spamwhitelisted} = $message->{spamwhitelisted};
    $msg{spamblacklisted} = $message->{spamblacklisted};
    $msg{sascore} = $message->{sascore};
    $msg{spamreport} = $spamreport;
    $msg{ismcp} = $message->{ismcp};
    $msg{ishighmcp} = $message->{ishighmcp};
    $msg{issamcp} = $message->{issamcp};
    $msg{mcpwhitelisted} = $message->{mcpwhitelisted};
    $msg{mcpblacklisted} = $message->{mcpblacklisted};
    $msg{mcpsascore} = $mcpsascore;
    $msg{mcpreport} = $mcpreport;
    $msg{virusinfected} = $message->{virusinfected};
    $msg{nameinfected} = $message->{nameinfected};
    $msg{otherinfected} = $message->{otherinfected};
    $msg{reports} = $reports;
    $msg{hostname} = $hostname;
    $msg{date} = $date;
    $msg{"time"} = $time;
    $msg{headers} = join("\n", @{$message->{headers}});
    $msg{quarantined} = $quarantined;

mmgomess commented 7 years ago

Sorry, but how do you do this? I'm a simple mortal.rsrs

Solution two(don't need to restart MySQL):

1.change the table and column encoding to utf8mb4

2.specifying characterEncoding=UTF-8 in the jdbc connector, cause the jdbc connector doesn't suport utf8mb4.

3.write your sql statment like this(need to add allowMultiQueries=true to jdbc connector):

'SET NAMES utf8mb4;INSERT INTO Mytable ...';

this will make sure each connection to the server, character_set_client,character_set_connection,character_set_results are utf8mb4.Also see charset-connection

2017-01-29 12:54 GMT-02:00 Stéphane notifications@github.com:

Line 266 in MailWatch.om:

# Place all data into %msg
my %msg;
$msg{timestamp} = $timestamp;
$msg{id} = $message->{id};
$msg{size} = $message->{size};
$msg{from} = $message->{from};
$msg{from_domain} = $message->{fromdomain};
$msg{to} = join(",", @{$message->{to}});
$msg{to_domain} = $todomain;
$msg{subject} = fix_latin($message->{utf8subject});
$msg{clientip} = $clientip;
$msg{archiveplaces} = join(",", @{$message->{archiveplaces}});
$msg{isspam} = $message->{isspam};
$msg{ishigh} = $message->{ishigh};
$msg{issaspam} = $message->{issaspam};
$msg{isrblspam} = $message->{isrblspam};
$msg{spamwhitelisted} = $message->{spamwhitelisted};
$msg{spamblacklisted} = $message->{spamblacklisted};
$msg{sascore} = $message->{sascore};
$msg{spamreport} = $spamreport;
$msg{ismcp} = $message->{ismcp};
$msg{ishighmcp} = $message->{ishighmcp};
$msg{issamcp} = $message->{issamcp};
$msg{mcpwhitelisted} = $message->{mcpwhitelisted};
$msg{mcpblacklisted} = $message->{mcpblacklisted};
$msg{mcpsascore} = $mcpsascore;
$msg{mcpreport} = $mcpreport;
$msg{virusinfected} = $message->{virusinfected};
$msg{nameinfected} = $message->{nameinfected};
$msg{otherinfected} = $message->{otherinfected};
$msg{reports} = $reports;
$msg{hostname} = $hostname;
$msg{date} = $date;
$msg{"time"} = $time;
$msg{headers} = join("\n", @{$message->{headers}});
$msg{quarantined} = $quarantined;

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mailwatch/1.2.0/issues/430#issuecomment-275918702, or mute the thread https://github.com/notifications/unsubscribe-auth/AJSnk4G7fg-9SRsOS8OE1I0Okqg3H10Xks5rXKgugaJpZM4Lq9fW .

mmgomess commented 7 years ago

P.S. I´m using Ubuntu 16.04 LTS

2017-01-29 12:57 GMT-02:00 Marcelo Machado mmgomess@gmail.com:

Sorry, but how do you do this? I'm a simple mortal.rsrs

Solution two(don't need to restart MySQL):

1.change the table and column encoding to utf8mb4

2.specifying characterEncoding=UTF-8 in the jdbc connector, cause the jdbc connector doesn't suport utf8mb4.

3.write your sql statment like this(need to add allowMultiQueries=true to jdbc connector):

'SET NAMES utf8mb4;INSERT INTO Mytable ...';

this will make sure each connection to the server, character_set_client,character_set_connection,character_set_results are utf8mb4.Also see charset-connection

2017-01-29 12:54 GMT-02:00 Stéphane notifications@github.com:
Line 266 in MailWatch.om:
# Place all data into %msg
my %msg;
$msg{timestamp} = $timestamp;
$msg{id} = $message->{id};
$msg{size} = $message->{size};
$msg{from} = $message->{from};
$msg{from_domain} = $message->{fromdomain};
$msg{to} = join(",", @{$message->{to}});
$msg{to_domain} = $todomain;
$msg{subject} = fix_latin($message->{utf8subject});
$msg{clientip} = $clientip;
$msg{archiveplaces} = join(",", @{$message->{archiveplaces}});
$msg{isspam} = $message->{isspam};
$msg{ishigh} = $message->{ishigh};
$msg{issaspam} = $message->{issaspam};
$msg{isrblspam} = $message->{isrblspam};
$msg{spamwhitelisted} = $message->{spamwhitelisted};
$msg{spamblacklisted} = $message->{spamblacklisted};
$msg{sascore} = $message->{sascore};
$msg{spamreport} = $spamreport;
$msg{ismcp} = $message->{ismcp};
$msg{ishighmcp} = $message->{ishighmcp};
$msg{issamcp} = $message->{issamcp};
$msg{mcpwhitelisted} = $message->{mcpwhitelisted};
$msg{mcpblacklisted} = $message->{mcpblacklisted};
$msg{mcpsascore} = $mcpsascore;
$msg{mcpreport} = $mcpreport;
$msg{virusinfected} = $message->{virusinfected};
$msg{nameinfected} = $message->{nameinfected};
$msg{otherinfected} = $message->{otherinfected};
$msg{reports} = $reports;
$msg{hostname} = $hostname;
$msg{date} = $date;
$msg{"time"} = $time;
$msg{headers} = join("\n", @{$message->{headers}});
$msg{quarantined} = $quarantined;
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mailwatch/1.2.0/issues/430#issuecomment-275918702, or mute the thread https://github.com/notifications/unsubscribe-auth/AJSnk4G7fg-9SRsOS8OE1I0Okqg3H10Xks5rXKgugaJpZM4Lq9fW .

branko77 commented 7 years ago

So am I :-)

On Jan 29, 2017 15:57, "mmgomess" notifications@github.com wrote:

Sorry, but how do you do this? I'm a simple mortal.rsrs

Solution two(don't need to restart MySQL):

1.change the table and column encoding to utf8mb4

2.specifying characterEncoding=UTF-8 in the jdbc connector, cause the jdbc connector doesn't suport utf8mb4.

3.write your sql statment like this(need to add allowMultiQueries=true to jdbc connector):

'SET NAMES utf8mb4;INSERT INTO Mytable ...';

this will make sure each connection to the server, character_set_client,character_set_connection,character_set_results are utf8mb4.Also see charset-connection

2017-01-29 12:54 GMT-02:00 Stéphane notifications@github.com:

Line 266 in MailWatch.om:

Place all data into %msg

my %msg; $msg{timestamp} = $timestamp; $msg{id} = $message->{id}; $msg{size} = $message->{size}; $msg{from} = $message->{from}; $msg{from_domain} = $message->{fromdomain}; $msg{to} = join(",", @{$message->{to}}); $msg{to_domain} = $todomain; $msg{subject} = fix_latin($message->{utf8subject}); $msg{clientip} = $clientip; $msg{archiveplaces} = join(",", @{$message->{archiveplaces}}); $msg{isspam} = $message->{isspam}; $msg{ishigh} = $message->{ishigh}; $msg{issaspam} = $message->{issaspam}; $msg{isrblspam} = $message->{isrblspam}; $msg{spamwhitelisted} = $message->{spamwhitelisted}; $msg{spamblacklisted} = $message->{spamblacklisted}; $msg{sascore} = $message->{sascore}; $msg{spamreport} = $spamreport; $msg{ismcp} = $message->{ismcp}; $msg{ishighmcp} = $message->{ishighmcp}; $msg{issamcp} = $message->{issamcp}; $msg{mcpwhitelisted} = $message->{mcpwhitelisted}; $msg{mcpblacklisted} = $message->{mcpblacklisted}; $msg{mcpsascore} = $mcpsascore; $msg{mcpreport} = $mcpreport; $msg{virusinfected} = $message->{virusinfected}; $msg{nameinfected} = $message->{nameinfected}; $msg{otherinfected} = $message->{otherinfected}; $msg{reports} = $reports; $msg{hostname} = $hostname; $msg{date} = $date; $msg{"time"} = $time; $msg{headers} = join("\n", @{$message->{headers}}); $msg{quarantined} = $quarantined;

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mailwatch/1.2.0/issues/430#issuecomment-275918702, or mute the thread https://github.com/notifications/unsubscribe-auth/AJSnk4G7fg- 9SRsOS8OE1I0Okqg3H10Xks5rXKgugaJpZM4Lq9fW .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mailwatch/1.2.0/issues/430#issuecomment-275918854, or mute the thread https://github.com/notifications/unsubscribe-auth/AXbXsHtzlfvt17X1ENTXBiFHYGmO2Jraks5rXKjHgaJpZM4Lq9fW .

Skywalker-11 commented 7 years ago

In the new version where we use mysqli the charset should already be set to utf8 or utf8mb4 database.php lines 55-60

spec1re commented 7 years ago

This morning MailWatchSQL just stopped logging to MySQL and crashed:

Could not use Custom Function code MailScanner::CustomConfig::InitMailWatchLogging, it could not be "eval"ed. Make sure the module is correct with perl -wc (Error: DBD::mysql::st execute failed: Incorrect string value: '\xFCberne...' for column 'headers' at row 1 at /usr/local/lib/MailScanner/MailScanner/CustomFunctions/MailWatch.pm line 173, <CLIENT> line 60.)

xFC = ü

Logfile in ANSI encoding

grafik

Logfile in UTF-8 encoding

grafik

This time its not a emoji but a umlaut (äüö) in the subject problem, not with all mails just with this one which looks like coming in ANSI encoded.

Maybe MailWatch needs some kind of utf8mb4 compliant check the subject before it unsuccessfully tries, to write it to the database and thus just stops working?

branko77 commented 7 years ago

I made changes in my.cnf but there is still crash after sending that email with subject that @spec1re suggested. Here is my.cnf:

[mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock character-set-client-handshake = FALSE character-set-server = utf8mb4 collation-server = utf8mb4_unicode_ci symbolic-links=0 sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES

[mysqld_safe] log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid

[client] default-character-set = utf8mb4

[mysql] default-character-set = utf8mb4

branko77 commented 7 years ago

And in a moment of crash I got this message on my CLI: Could not use Custom Function code MailScanner::CustomConfig::InitMailWatchLogging, it could not be "eval"ed. Make sure the module is correct with perl -wc (Error: DBD::mysql::st execute failed: Incorrect string value: '\xF0\x9F\x92\x98\xC2\xA0...' for column 'subject' at row 1 at /usr/share/MailScanner/perl/custom/MailWatch.pm line 138, <CLIENT> line 620. ) at /usr/share/MailScanner/perl/MailScanner/Config.pm line 1053.

asuweb commented 7 years ago

I'm assuming you have Encoding::FixLatin perl module installed?

Mailscanner lint usually complains if you haven't, but might be worth checking

branko77 commented 7 years ago

Yes Encoding::FixLatin is installed

MailScanner --lint Trying to setlogsock(unix)

Reading configuration file /etc/MailScanner/MailScanner.conf Read 1501 hostnames from the phishing whitelist Read 12730 hostnames from the phishing blacklists Config: calling custom init function SQLBlacklist MailWatch: Starting up MailWatch SQL Blacklist MailWatch: Read 311 blacklist entries Config: calling custom init function SQLWhitelist MailWatch: Starting up MailWatch SQL Whitelist MailWatch: Read 43 whitelist entries

Checking version numbers... Version number in MailScanner.conf (5.0.3) is correct.

Your envelope_sender_header in spamassassin.conf is correct. MailScanner setting GID to (5002) MailScanner setting UID to (89)

Checking for SpamAssassin errors (if you use it)... Using SpamAssassin results cache Connected to SpamAssassin cache database SpamAssassin reported no errors. Connected to Processing Attempts Database Created Processing Attempts Database successfully There are 0 messages in the Processing Attempts Database Using locktype = posix MailScanner.conf says "Virus Scanners = clamav" Found these virus scanners installed: clamavmodule

Filename Checks: Windows/DOS Executable (1 eicar.com) Other Checks: Found 1 problems Virus and Content Scanning: Starting 1.message: Eicar-Test-Signature FOUND

./1/eicar.com: Eicar-Test-Signature FOUND

Virus Scanning: ClamAV found 2 infections Infected message 1 came from 10.1.1.1 Virus Scanning: Found 2 viruses

Virus Scanner test reports: ClamAV said "eicar.com contains Eicar-Test-Signature"

If any of your virus scanners (clamavmodule) are not listed there, you should check that they are installed correctly and that MailScanner is finding them correctly via its virus.scanners.conf. Config: calling custom end function SQLBlacklist MailWatch: Closing down MailWatch SQL Blacklist Config: calling custom end function SQLWhitelist MailWatch: Closing down MailWatch SQL Whitelist

mmgomess commented 7 years ago

Same with me. I did make the changes in my.cnf but the MailWatch was stopped.

2017-01-30 8:22 GMT-02:00 branko77 notifications@github.com:

Yes Encoding::FixLatin is installed

MailScanner --lint Trying to setlogsock(unix)

Reading configuration file /etc/MailScanner/MailScanner.conf Read 1501 hostnames from the phishing whitelist Read 12730 hostnames from the phishing blacklists Config: calling custom init function SQLBlacklist MailWatch: Starting up MailWatch SQL Blacklist MailWatch: Read 311 blacklist entries Config: calling custom init function SQLWhitelist MailWatch: Starting up MailWatch SQL Whitelist MailWatch: Read 43 whitelist entries

Checking version numbers... Version number in MailScanner.conf (5.0.3) is correct.

Your envelope_sender_header in spamassassin.conf is correct. MailScanner setting GID to (5002) MailScanner setting UID to (89) Checking for SpamAssassin errors (if you use it)... Using SpamAssassin results cache Connected to SpamAssassin cache database SpamAssassin reported no errors. Connected to Processing Attempts Database Created Processing Attempts Database successfully There are 0 messages in the Processing Attempts Database Using locktype = posix MailScanner.conf says "Virus Scanners = clamav" Found these virus scanners installed: clamavmodule

Filename Checks: Windows/DOS Executable (1 eicar.com) Other Checks: Found 1 problems Virus and Content Scanning: Starting 1.message: Eicar-Test-Signature FOUND

./1/eicar.com: Eicar-Test-Signature FOUND Virus Scanning: ClamAV found 2 infections Infected message 1 came from 10.1.1.1 Virus Scanning: Found 2 viruses

Virus Scanner test reports: ClamAV said "eicar.com contains Eicar-Test-Signature"

If any of your virus scanners (clamavmodule) are not listed there, you should check that they are installed correctly and that MailScanner is finding them correctly via its virus.scanners.conf. Config: calling custom end function SQLBlacklist MailWatch: Closing down MailWatch SQL Blacklist Config: calling custom end function SQLWhitelist MailWatch: Closing down MailWatch SQL Whitelist

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mailwatch/1.2.0/issues/430#issuecomment-276026248, or mute the thread https://github.com/notifications/unsubscribe-auth/AJSnk9zyHWTKLu89qjkQ1G3KOgxwuEstks5rXbnSgaJpZM4Lq9fW .

spec1re commented 7 years ago

[root@MS~]$ mailscanner -v | grep missing
missing SAVI

Encoding::FixLatin is installed

Trying to setlogsock(unix)  
Reading configuration file /usr/local/etc/MailScanner/MailScanner.conf  
Read 1501 hostnames from the phishing whitelist     
Read 13140 hostnames from the phishing blacklists   
Config: calling custom init function SQLWhitelist   
MailWatch: Starting up MailWatch SQL Whitelist  
MailWatch: Read 28 whitelist entries    
Config: calling custom init function MailWatchLogging   
MailWatch: Started MailWatch SQL Logging child 
Config: calling custom init function SQLBlacklist   
MailWatch: Starting up MailWatch SQL Blacklist 
MailWatch: Read 1 blacklist entries     
Checking version numbers...     
Version number in MailScanner.conf (4.85.2) is correct.     
Your envelope_sender_header in spam.assassin.prefs.conf is correct. 
MailScanner setting GID to (9595) 
MailScanner setting UID to (125)    
Checking for SpamAssassin errors (if you use it)...     
Using SpamAssassin results cache    
Connected to SpamAssassin cache database    
SpamAssassin reported no errors.    
Connected to Processing Attempts Database   
Created Processing Attempts Database successfully   
There are 0 messages in the Processing Attempts Database    
Using locktype = posix 
MailScanner.conf says "Virus Scanners = clamd" 
Found these virus scanners installed: clamavmodule, clamd   
===========================================================================     
Filename Checks: Windows/DOS Executable (1 eicar.com)   
Other Checks: Found 1 problems 
Virus and Content Scanning: Starting    
Clamd::INFECTED::Eicar-Test-Signature :: ./1/   
Clamd::INFECTED:: Eicar-Test-Signature :: ./1/eicar.com     
Virus Scanning: Clamd found 2 infections    
Infected message 1 came from 10.1.1.1   
Virus Scanning: Found 2 viruses     
===========================================================================     
Virus Scanner test reports:     
Clamd said "eicar.com was infected: Eicar-Test-Signature" 
If any of your virus scanners (clamavmodule,clamd)  
are not listed there, you should check that they are installed correctly    
and that MailScanner is finding them correctly via its virus.scanners.conf.     
Config: calling custom end function SQLWhitelist    
MailWatch: Closing down MailWatch SQL Whitelist     
Config: calling custom end function MailWatchLogging    
Config: calling custom end function SQLBlacklist    
MailWatch: Closing down MailWatch SQL Blacklist

Looks good so far.

spec1re commented 7 years ago

Here the mail, which will crash MailWatchSQL:

crash.eml.txt

Just try:

swaks -f from@example.com -t me@localhost.de -s 10.1.1.1 -d /tmp/spamassassin/crash.eml

and watch MailWatchSQL goes down with:

Could not use Custom Function code MailScanner::CustomConfig::InitMailWatchLogging, it could not be "eval"ed. Make sure the module is correct with perl -wc (Error: DBD::mysql::st execute failed: Incorrect string value: '\xFCberne...' for column 'headers' at row 1 at /usr/local/lib/MailScanner/MailScanner/CustomFunctions/MailWatch.pm line 173, line 60.)

X-PHP-Originating-Script: 1597366:cron_castingangebote.php

I guess bad encoding by php?

stefaweb commented 7 years ago

Pushed #425. Read comments in PR.

Try only on dev server please.

spec1re commented 7 years ago

425 starts with errors

Starting mailscanner. Could not use Custom Function code /usr/local/lib/MailScanner/MailScanner/CustomFunctions/SQLSpamSettings.pm, it could not be "require"d. Make sure the last line is "1;" and the module is correct with perl -wc (Error: Global symbol "$dbh" requires explicit package name (did you forget to declare "my $dbh"?) at /usr/local/lib/MailScanner/MailScanner/CustomFunctions/SQLSpamSettings.pm line 62. Global symbol "$dbh" requires explicit package name (did you forget to declare "my $dbh"?) at /usr/local/lib/MailScanner/MailScanner/CustomFunctions/SQLSpamSettings.pm line 66. Global symbol "$dbh" requires explicit package name (did you forget to declare "my $dbh"?) at /usr/local/lib/MailScanner/MailScanner/CustomFunctions/SQLSpamSettings.pm line 69. Global symbol "$dbh" requires explicit package name (did you forget to declare "my $dbh"?) at /usr/local/lib/MailScanner/MailScanner/CustomFunctions/SQLSpamSettings.pm line 70. Compilation failed in require at /usr/local/lib/MailScanner/MailScanner/Config.pm line 760. ) at /usr/local/lib/MailScanner/MailScanner/Config.pm line 762.

asuweb commented 7 years ago

@spec1re - I've just tested the email you provided, and it works without issue on the newly built servers

spec1re commented 7 years ago

@asuweb - Can you please share your MySQL etc. configs with me?

asuweb commented 7 years ago

For debian or centos?

asuweb commented 7 years ago

Debian config below - these are the stock config, nothing has been modified

cat: /etc/mysql/conf.d: Is a directory
# Automatically generated for Debian scripts. DO NOT TOUCH!
[client]
host     = localhost
user     = debian-sys-maint
password = redacted
socket   = /var/run/mysqld/mysqld.sock
[mysql_upgrade]
host     = localhost
user     = debian-sys-maint
password = redacted
socket   = /var/run/mysqld/mysqld.sock
basedir  = /usr
#!/bin/bash
#
# This script is executed by "/etc/init.d/mysql" on every (re)start.
#
# Changes to this file will be preserved when updating the Debian package.
#

source /usr/share/mysql/debian-start.inc.sh

MYSQL="/usr/bin/mysql --defaults-file=/etc/mysql/debian.cnf"
MYADMIN="/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf"
MYUPGRADE="/usr/bin/mysql_upgrade --defaults-extra-file=/etc/mysql/debian.cnf"
MYCHECK="/usr/bin/mysqlcheck --defaults-file=/etc/mysql/debian.cnf"
MYCHECK_SUBJECT="WARNING: mysqlcheck has found corrupt tables"
MYCHECK_PARAMS="--all-databases --fast --silent"
MYCHECK_RCPT="root"

## Checking for corrupt, not cleanly closed and upgrade needing tables.

# The following commands should be run when the server is up but in background
# where they do not block the server start and in one shell instance so that
# they run sequentially. They are supposed not to echo anything to stdout.
# If you want to disable the check for crashed tables comment
# "check_for_crashed_tables" out.
# (There may be no output to stdout inside the background process!)

# Need to ignore SIGHUP, as otherwise a SIGHUP can sometimes abort the upgrade
# process in the middle.
trap "" SIGHUP
(
  upgrade_system_tables_if_necessary;
  check_root_accounts;
  check_for_crashed_tables;
) >&2 &

exit 0
#
# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html

# This will be passed to all mysql clients
# It has been reported that passwords should be enclosed with ticks/quotes
# escpecially if they contain "#" chars...
# Remember to edit /etc/mysql/debian.cnf when changing the socket location.
[client]
port            = 3306
socket          = /var/run/mysqld/mysqld.sock

# Here is entries for some specific programs
# The following values assume you have at least 32M ram

# This was formally known as [safe_mysqld]. Both versions are currently parsed.
[mysqld_safe]
socket          = /var/run/mysqld/mysqld.sock
nice            = 0

[mysqld]
#
# * Basic Settings
#
user            = mysql
pid-file        = /var/run/mysqld/mysqld.pid
socket          = /var/run/mysqld/mysqld.sock
port            = 3306
basedir         = /usr
datadir         = /var/lib/mysql
tmpdir          = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address            = 127.0.0.1
#
# * Fine Tuning
#
key_buffer              = 16M
max_allowed_packet      = 16M
thread_stack            = 192K
thread_cache_size       = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover         = BACKUP
#max_connections        = 100
#table_cache            = 64
#thread_concurrency     = 10
#
# * Query Cache Configuration
#
query_cache_limit       = 1M
query_cache_size        = 16M
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file        = /var/log/mysql/mysql.log
#general_log             = 1
#
# Error log - should be very few entries.
#
log_error = /var/log/mysql/error.log
#
# Here you can see queries with especially long duration
#slow_query_log_file = /var/log/mysql/mysql-slow.log
#slow_query_log      = 1
#long_query_time = 2
#log_queries_not_using_indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
#       other settings you may need to change.
#server-id              = 1
#log_bin                        = /var/log/mysql/mysql-bin.log
expire_logs_days        = 10
max_binlog_size         = 100M
#binlog_do_db           = include_database_name
#binlog_ignore_db       = include_database_name
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem

[mysqldump]
quick
quote-names
max_allowed_packet      = 16M

[mysql]
#no-auto-rehash # faster start of mysql but no tab completition

[isamchk]
key_buffer              = 16M

#
# * IMPORTANT: Additional settings that can override those from this file!
#   The files must end with '.cnf', otherwise they'll be ignored.
#
!includedir /etc/mysql/conf.d/

root@devdebian:/home/debian# cat /etc/mysql/conf.d/*
# MariaDB-specific config file.
# Read by /etc/mysql/my.cnf

[client]
# Default is Latin1, if you need UTF-8 set this (also in server section)
#default-character-set = utf8

[mysqld]
#
# * Character sets
#
# Default is Latin1, if you need UTF-8 set all this (also in client section)
#
#character-set-server  = utf8
#collation-server      = utf8_general_ci
#character_set_server   = utf8
#collation_server       = utf8_general_ci
[mysqld_safe]
skip_log_error
syslog
[mariadb]
# See https://mariadb.com/kb/en/how-to-enable-tokudb-in-mariadb/
# for instructions how to enable TokuDB
#
# See https://mariadb.com/kb/en/tokudb-differences/ for differences
# between TokuDB in MariaDB and TokuDB from http://www.tokutek.com/

#plugin-load-add=ha_tokudb.so

stefaweb commented 7 years ago

Pushed new version of SQLSpamSettings.pm in #425 (variables was not declared at the right place).

spec1re commented 7 years ago

@stefaweb - It's working now without the settings in my.cnf 👍

stefaweb commented 7 years ago

Great!

spec1re commented 7 years ago

I just need to fix the ANSI encoded umlauts in subject crash, any hint for me where to look for?

stefaweb commented 7 years ago

Pushed new version of SQLSpamSettings.pm in #425 (new log messages and one error corrected).

spec1re commented 7 years ago

425 starts without errors

Could not use Custom Function code MailScanner::CustomConfig::InitMailWatchLogging, it could not be "eval"ed. Make sure the module is correct with perl -wc (Error: DBD::mysql::st execute failed: Incorrect string value: '\xFCberne...' for column 'headers' at row 1 at /usr/local/lib/MailScanner/MailScanner/CustomFunctions/MailWatch.pm line 216, line 436.) at /usr/local/lib/MailScanner/MailScanner/Config.pm line 1061.

stefaweb commented 7 years ago

If you have badly coded utf8 record in maillog, you must delete them.

spec1re commented 7 years ago

grafik V-DAY work for me as well, just ANSI encoded umlauts like \xFC killing me. :P

endelwar commented 7 years ago

Could you guys report the dbd-mysql perl library version present on problematic servers? Looking at the changelog (https://github.com/perl5-dbi/DBD-mysql/blob/4867cab3fc47f71211660126ff15d928424fb912/Changes#L21-L24) seems that there are some problem with utf8 encoding

spec1re commented 7 years ago

[root@MS~]$ perl -MDBD::mysql -e 'print $DBD::mysql::VERSION'

4.041_1

I run 4.041_1 dev-version since 3 hours, before it was 4.041.

spec1re commented 7 years ago

@asuweb - Which version of MailScanner is running on your built servers?

stefaweb commented 7 years ago

@endelwar : On Debian 8 we have package libdbd-mysql-perl 4.028-2+deb8u2 (4.028 with CVE patches)

asuweb commented 7 years ago

@spec1re - 5.0.3

spec1re commented 7 years ago

Here on FreeBSD I'm still at MailScanner Version 4.85.2. How do the subject, of the crash.eml, look like in the mail.log if you open it in notepad++?

grafik

Can you do me a favor and downgrade to MSv4 and try the crash mail again? 9_9

branko77 commented 7 years ago

Hm... I need a bit of help from you guys... Maybe you remeber, when I set up my MW i had a problem with: "MailWatch: Error: unexpected connection from 192.168.3.1 at /usr/share/MailScanner/perl/custom/MailWatch.pm line 114." And you advised me to change from 127.0.0.1 in 192.168.3.1 in that line and that resolved the problem.

Now I downloaded new MailWatch.pm file that you guys edited yesterday and changed from 127.0.0.1 in 192.168.3.1 on line 159 but I still have this error: MailWatch: Error: unexpected connection from 192.168.3.1 at /usr/share/MailScanner/perl/custom/MailWatch.pm line 159.

Any idea about this?

mailwatch / MailWatch

MailWatch stopped logging in to MySQL #430

Place all data into %msg

425 starts with errors

425 starts without errors