details.php Message ID '1r6Bwm-000000002h3-2n4V' not found!

sjaak327 commented 8 months ago

Issue summary

Write a short summary of the issue, trying to pinpoint it as much as possible. Mention the URL where this bug occurs, if applicable. Please include full error messages and screenshots if possible.

Steps to reproduce

Click on the message details
I get Message ID '1r6Bwm-000000002h3-2n4V' not found!

Expected result

it should display the message details

Actual result

Message ID '1r6Bwm-000000002h3-2n4V' not found!

Installation

Version and method

MailWatch Version: [ stable 1.2.x | RC-x | master | develop ]
Install type: [ GitHub checkout | Zip/tarball install | Installer (beta) ]
Updated from an older MailWatch or fresh install: [ Fresh install | Upgrade ] (from version [ version ])

Server configuration

Operation System: [ Debian | Ubuntu | CentOS | Other ] version [ version ] Ubuntu 22.04.3 LTS
PHP version: [ 5.3 | 5.4 | 5.5 | 5.6 | 7.0 | 7.1 ] 8.2.12
Database: [ MySQL | MariaDB ] version [ version ] MySQL Version: 8.0.35-0ubuntu0.22.04.1
Mail Server: [ Postfix | Exim | Sendmail ] exim 4.97 self compiled. using 23 char msgid
Used web server: [ Apache | Nginx ] version [ version ] apache2

Client configuration

Operation System: [ operation system ] version [ version ]
Browser: [ browser name and version ]

I guess the problem is the longer exim message ID, I already updated the msgid in mtalog to be 23 instead of 20, this fixed exim being unable to record the message in the dB for status delivery.

Mailscanner shows this:

MailWatch: Starting up MailWatch SQL Allowlist MailWatch: Read 14 allowlist entries MailWatch: Blocklist refresh time reached MailWatch: Starting up MailWatch SQL Blocklist MailWatch: Read 5 blocklist entries Uninfected: Delivered 1 messages Deleted 1 messages from processing-database MailWatch: Could not extract Message-ID for 1r6Bwm-000000002h3-2n4V MailWatch: Logging message 1r6Bwm-000000002h3-2n4V to SQL MailWatch SQL[1565]: 1r6Bwm-000000002h3-2n4V: Logged to MailWatch SQL

it seems it does store the message into the mailscanner database in maillog but the details.php does return nothing.

I guess this PHP fiile neeeds adujsting somehwere.

sjaak327 commented 8 months ago

changing exim to retain_sender at least got rid of the error message "could not extract message-id) but the issues remains.

sjaak327 commented 8 months ago

I have for now recompile exim at version 4.96.2 which still uses smaller msgid length. than this issue goes away. I guess you need to change some thing to commodate for exim's 23 char msgid starting at 4.97

shawniverson commented 8 months ago

https://github.com/MailScanner/v5/issues/672

sjaak327 commented 8 months ago

I applied the fix in that link. does not work. Yes mailscanner no longer complains:

Nov 24 18:36:41 jvangentsvr4 MailScanner[29129]: New Batch: Scanning 1 messages, 8177 bytes Nov 24 18:36:42 jvangentsvr4 MailScanner[29129]: Virus and Content Scanning: Starting Nov 24 18:36:45 jvangentsvr4 MailScanner[29129]: Uninfected: Delivered 1 messages Nov 24 18:36:45 jvangentsvr4 MailScanner[29129]: Deleted 1 messages from processing-database Nov 24 18:36:46 jvangentsvr4 MailScanner[29129]: MailWatch: Logging message 1r6a6b-000000007d6-2ggd to SQL Nov 24 18:36:46 jvangentsvr4 MailWatch SQL[29116]: 1r6a6b-000000007d6-2ggd: Logged to MailWatch SQL

But the same end result, it is visible in the message list and as soon as I click details, I get the error Message ID '1r6a6b-000000007d6-2ggd' not found!

It would be nice if someone could pinpoint where exactly this goes wrong. As downgrading to 4.96.2 means opendmarc no longer works correctly.

sjaak327 commented 8 months ago

As an addon, these message also remain in the outQ despite me doing a queue run in exim to flush the queue (my exchange server was down for some disk mantenance). There were four messages in that queue, two received with exim 4.96.2 and two with 4.97,. all four have been delivered, but the last two remain in que viewer and in de outq table in the db.

shawniverson commented 8 months ago

@sjaak327 Are two of the messages stuck in the out queue because of switching exim versions while they were still in the queue, so they aren't getting removed properly?

sjaak327 commented 8 months ago

Not sure. I just checked, they were not delivered by exim aparrently as the files are still physically in /var/spool/exim4/input.

I am doing more disk maintenance as my raid1 here failed, replaced the faulty disk but now the other disk is having bad sectors, I put the exchange VM down, as it is using that volume, running checkdisk/ /.f /r now, that will take some hours, after that I will look at what happens to them.

sjaak327 commented 8 months ago

I applied your fix to functions.php and that seems to now fully fix the issue, togehter with the four changes to EximDisk.pm for mailscanner.

sjaak327 commented 8 months ago

Now the outqueue is empty. no more files left, so I think it now works correctly.

tony451 commented 6 months ago

The change to functions.php suggested by shawniverson #1294 still needs to be applied to Mailwatch to support new message ids in Exim 4.97.

mailwatch / MailWatch