Generalize detection of dat files

[shawniverson]

Fixes #554

[janarzz]

I dont see any changes here: https://github.com/MailScanner/v5/pull/556/commits/cfb0221b51ad5e2a4b17e27d898370b7f1952f74

[shawniverson]

@janarzz This code is copied to these lines, you need to add the code here at line 900.

[janarzz]

I dont understand..line 900 is empty. I already have these lines in the configuration. https://pilv.addit.ee/index.php/s/p5WEqz5QYXNPSLJ

[msapiro]

@janarzz Go to https://github.com/MailScanner/v5/commit/cfb0221b51ad5e2a4b17e27d898370b7f1952f74

What that says is add these lines

      if (MailScanner::Config::Value('aignoredatexecutable', $message) =~ /1/ && $attach =~ /\.(?:dat|cdr)$/) {
        ## Will prevent to quarantine email if MS Office/Corel
        ## attachment contains a .dat file
        ## .dat files are detected as executable in some instances
        MailScanner::Log::InfoLog("Skipping archive .dat file type check (prevent wrong executable type)");
        next;
      }

between lines 900 and line 901.

[janarzz]

Hi, i get these warning message:

Our e-mail content detector has just been triggered by a message you sent: To: janar@addit.ee Subject: corel Date: Thu Oct 14 10:17:26 2021

One or more of the attachments (P11_Liela balva.cdr, 5page1.dat) are on the list of unacceptable attachments for this site and will not have been delivered.

Consider renaming the files to avoid this constraint.

The virus detector said this about the message: Report: Report: MailScanner: No programs allowed (5page1.dat)

And this..

Warning: This message has had one or more attachments removed Warning: (5page1.dat, P11_Liela balva.cdr). Warning: Please read the "yoursite-Attachment-Warning.txt" attachment(s) for more information.

Still mailscanner blocked this file.

[shawniverson]

@janarzz We need to ensure the file is exact. Maybe download https://raw.githubusercontent.com/MailScanner/v5/cfb0221b51ad5e2a4b17e27d898370b7f1952f74/common/usr/share/MailScanner/perl/MailScanner/SweepOther.pm and place it in /usr/share/MailScanner/perl/MailScanner (after backing up your old SweepOther.pm)

[janarzz]

Hi, all is same. I replace my SweepOther.pm, but still same error and files is blocked.

Warning: This message has had one or more attachments removed Warning: (P11_Liela balva.cdr, 5page1.dat). Warning: Please read the "yoursite-Attachment-Warning.txt" attachment(s) for more information.

Our e-mail content detector has just been triggered by a message you sent: To: janar@addit.ee Subject: Corel test Date: Fri Oct 15 08:24:24 2021

One or more of the attachments (5page1.dat, P11_Liela balva.cdr) are on the list of unacceptable attachments for this site and will not have been delivered.

Consider renaming the files to avoid this constraint.

The virus detector said this about the message: Report: Report: MailScanner: No programs allowed (5page1.dat)

[shawniverson]

@janarzz can you capture your mail log again and share?

[janarzz]

New maillog here: https://pilv.addit.ee/index.php/s/4H8THQo9Scj6mzc

[shawniverson]

@janarzz That doesn't look like it was blocked, but you are still having issues?

[janarzz]

When i tryng send this file, then i get this messages: Our e-mail content detector has just been triggered by a message you sent: To: janar@addit.ee Subject: Corel Test Date: Tue Oct 19 09:02:22 2021

One or more of the attachments (P11_Liela balva.cdr, 5page1.dat) are on the list of unacceptable attachments for this site and will not have been delivered.

Consider renaming the files to avoid this constraint.

The virus detector said this about the message: Report: Report: MailScanner: No programs allowed (5page1.dat)

Warning: This message has had one or more attachments removed Warning: (P11_Liela balva.cdr, 5page1.dat). Warning: Please read the "yoursite-Attachment-Warning.txt" attachment(s) for more information.

[shawniverson]

@janarzz any chance I could get my hands on this file?

[janarzz]

I sending this file your email.

[shawniverson]

@janarzz This is weird because when I run the file through it goes through successfully. I enabled Notify Senders to see if I would get the warning and I did not. I also set both file and file -i to test both branches of code.

[shawniverson]

I enabled some extra debugging and I see the dat files being skipped

Oct 19 07:32:02 smtp MailScanner[4835]: Unpacked Zip archive: nP11_Liela balva.cdr
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = masterPage.dat
Oct 19 07:32:02 smtp MailScanner[4835]: Skipping archive .dat file type check (prevent wrong executable type)
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = textinfo.xml
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = page1.dat
Oct 19 07:32:02 smtp MailScanner[4835]: Skipping archive .dat file type check (prevent wrong executable type)
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = page1.bmp
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = msg-4835-1.txt
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = color.xml
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = dataFileList.dat
Oct 19 07:32:02 smtp MailScanner[4835]: Skipping archive .dat file type check (prevent wrong executable type)
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = root.dat
Oct 19 07:32:02 smtp MailScanner[4835]: Skipping archive .dat file type check (prevent wrong executable type)
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = P11_Liela balva.cdr
Oct 19 07:32:02 smtp MailScanner[4835]: Skipping archive .dat file type check (prevent wrong executable type)
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = docPalette.xml
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = data1.dat
Oct 19 07:32:02 smtp MailScanner[4835]: Skipping archive .dat file type check (prevent wrong executable type)
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = thumbnail.bmp
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = links.xml
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = isocoated_v2_eci.icc
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = data2.dat
Oct 19 07:32:02 smtp MailScanner[4835]: Skipping archive .dat file type check (prevent wrong executable type)
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = Bitmaps.dat
Oct 19 07:32:02 smtp MailScanner[4835]: Skipping archive .dat file type check (prevent wrong executable type)
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = metadata.xml
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = srgb color space profile.icm
Oct 19 07:32:02 smtp MailScanner[4835]: Debug: File = document.cdss

[janarzz]

What is wrong my configuration ?

[shawniverson]

Have you restarted MailScanner recently?

[janarzz]

I restart mailscanner after any changes.

[shawniverson]

Can you send me your copy of /usr/share/MailScanner/perl/MailScanner/SweepOther.pm?

[janarzz]

I sendind this your email.

[shawniverson]

Well, I'm scratching my head...I'm still digging. It can't be SweepOther.pm that is triggering this at this point.

[janarzz]

You received the file ?

[shawniverson]

Yes, it is correct.

[janarzz]

Maybe must chnage another conf file or smth ?

[shawniverson]

Maybe, can you share me your MailScanner config?

[janarzz]

I sent

[shawniverson]

This is bizarre. Based on your config you shouldn't get a notification at all, let alone a blocked .dat file. It is almost as if you have another mailscanner altogether or a different config hiding out.

[janarzz]

No, i dont have another mailscanner and hiding config. Just from the beginning .dat file are blocked. I have tried add .dat allow filetype, but nothing. Still block.

[shawniverson]

I'm at a loss because I literally used your config on my instance and getting the correct behavior. I must be missing something.

[shawniverson]

@janarzz I'm going to build another mailscanner with the same version as yours and see if that makes any difference.

[janarzz]

Ok, for now i have: Mailscanner 5.4.1 MailWatch 1.2.10 ClamAV Version: 0.103.2

[shawniverson]

Refactor for user-defined extensions

[shawniverson]

Leaving this setting for just dat files, going to do a more generalized approach for other file types.