hMailServer bug with FETCHes on a message/rfc822 part

[RvdHout]

In my MarkAsJunk plugin config.inc.php

// Learning driver
// Use an external process such as sa-learn to learn from spam/ham messages. Default: null.
// Please see the README for more information
// $config['markasjunk_learning_driver'] = null;
$config['markasjunk_learning_driver'] = 'email_learn';

...

// email_learn Driver options
// --------------------------
// The email address that spam messages will be sent to
// The address can contain the following macros that will be expanded as follows:
//      %u is replaced with the username (from the session info)
//      %l is replaced with the local part of the username (if the username is an email address)
//      %d is replaced with the domain part of the username (if the username is an email address or default mail domain if not)
//      %i is replaced with the email address from the user's default identity
// If you do not want to send an email set this to null
//$config['markasjunk_email_spam'] = null;
//$config['markasjunk_email_spam'] = '%l+%d+spam@mydomain.nl';
$config['markasjunk_email_spam'] = 'spam@mydomain.nl';

// The email address that ham messages will be sent to
// The address can contain the following macros that will be expanded as follows:
//      %u is replaced with the username (from the session info)
//      %l is replaced with the local part of the username (if the username is an email address)
//      %d is replaced with the domain part of the username (if the username is an email address or default mail domain if not)
//      %i is replaced with the email address from the user's default identity
// If you do not want to send an email set this to null
//$config['markasjunk_email_ham'] = null;
//$config['markasjunk_email_ham'] = '%l+%d+ham@mydomain.nl';
$config['markasjunk_email_ham'] = 'ham@mydomain.nl';

// Should the spam/ham message be sent as an attachment
$config['markasjunk_email_attach'] = true;

// The email subject (when sending as attachment)
// The subject can contain the following macros that will be expanded as follows:
//      %u is replaced with the username (from the session info)
//      %l is replaced with the local part of the username (if the username is an email address)
//      %d is replaced with the domain part of the username (if the username is an email address or default mail domain if not)
//      %t is replaced with the type of message (spam/ham)
$config['markasjunk_email_subject'] = 'learn this message as %t';

...

As expected the spam/ham reported mail then comes in attached to a simple email message (with attachment shown inline), but when i open (in a new window) that attachment within roundcube it shows the original headers, and when i save the message as eml the headers are missing completely (roundcube 1.6.1)

See attachment for spam as received by the MarkAsJunk email-learn report (PS: i only anonymized my domain and client domain) learn this message as spam.zip

[RvdHout]

[RvdHout]

Same happens when i forward a message as message/rfc822 attachment to myself, once i open the message/rfc822 attachment it uses the headers of the parent message and not those from the original message, i think this is wrong. Or at least i would expect it to use the original message headers.

[alecpl]

I'm unable to reproduce with the message sample you provided in Roundcube 1.6-git.

We'll need more info. What plugins? What PHP version? What IMAP server? Could you provide imapdebug log? What are you imap* settings in Roundcube?

[RvdHout]

• php 8.0.28 (On IIS, Windows Server 2019)
• Server hMailServer
• imap_* settings in Roundcube all default, except:

$config['imap_host'] = 'localhost:143';
$config['imap_vendor'] = 'hmail';
$config['imap_debug'] = true;

// List of active plugins (in plugins/ directory)
$config['plugins'] = array(
    'zipdownload',
    '[hms_autoban](https://github.com/RvdHout/hms_autoban)',
    '[hms_autoreply](https://github.com/tunis78/roundcube-hms_autoreply)',
    '[hms_forwarding](https://github.com/tunis78/roundcube-hms_forwarding)',
    '[kolab_2fa](https://github.com/thomascube/roundcube-plugin-kolab-2fa)',
    'password',
    '[tls_icon_with_hmail](https://github.com/oszto90/Roundcube_TLS_Icon_with_Hmailserver)',
    'markasjunk',
);

imap_log contains steps displayed in graphics above imap.log

[RvdHout]

No difference if i only enable

$config['plugins'] = array(
    'zipdownload',
    'password',
    'markasjunk',
);

and/or comment the $config['imap_vendor'] directive

[alecpl]

Here's how it looks like on my server:

[13-May-2023 16:18:33.984465]: <ibgs6p48> [475E] C: A0006 UID FETCH 25 (BODY.PEEK[2.MIME])
[13-May-2023 16:18:34.016608]: <ibgs6p48> [475E] S: * 19 FETCH (UID 25 BODY[2.MIME] {399}
[13-May-2023 16:18:34.016856]: <ibgs6p48> [475E] S: Content-Transfer-Encoding: 8bit
Content-Type: message/rfc822;
 name="=?UTF-8?Q?Stable_expression/interference_cell_line_from_6_weeks!_C?=
 =?UTF-8?Q?lone_deliverables_also_available!=2Eeml?="
Content-Disposition: attachment;
 filename*0*=UTF-8''Stable%20expression%2Finterference%20cell%20line%20fro;
 filename*1*=m%206%20weeks!%20Clone%20deliverables%20also%20available!.eml;
 size=52510

[13-May-2023 16:18:34.016936]: <ibgs6p48> [475E] S: )
[13-May-2023 16:18:34.017010]: <ibgs6p48> [475E] S: A0006 OK Completed (0.000 sec)
[13-May-2023 16:18:34.049879]: <ibgs6p48> [475E] C: A0008 UID FETCH 25 (BODY.PEEK[2])
[13-May-2023 16:18:34.082636]: <ibgs6p48> [475E] S: * 19 FETCH (UID 25 BODY[2] {52506}
[13-May-2023 16:18:34.082910]: <ibgs6p48> [475E] S: Return-Path: <BOUNCE.E819D9.B57A22B0.134F09A.104BA7.bme@pmta605.dedicated.bmsend.com>
Delivered-To: chris@clientdomain.nl
X-hMailServer-Envelope-From: BOUNCE.E819D9.B57A22B0.134F09A.104BA7.bme@pmta605.dedicated.bmsend.com
X-Spam-Checker-Version: SpamAssassin 4.0.0-r1906050 (2022-12-17) on mail.mydomain.nl
X-Spam-Level: ***
...

So, my server treats the message/rfc822 part as any other part and returns what Roundcube expects, but it looks that hMailServer treats that part differently and returns the inner message parts.

I'd say it's a hMailServer bug, and I don't see how we could detect this situation. Trying to workaround this might not be that simple.

[RvdHout]

Thank you for investigating this, surely looks like a hmailserver bug then...

So if i understand right the difference is caused by what is returned from hmailserver when issuing the BODY.PEEK[2.MIME] and BODY.PEEK[2] commands?

[RvdHout]

$config['imap_vendor'] = 'hmail'; is never actually used, is it? I couldn't find a single reference to it in roundcube code....might be old leftover

Any idea what this setting used to do (in the past)? Guess it was there for a reason although it might be unrelated

[RvdHout]

@alecpl

Seem te be people that suggest it is roundcube issue as there is no such issue using clients like gmail or rainloop https://www.hmailserver.com/forum/viewtopic.php?f=8&p=247192

[alecpl]

It looks to me that these people do not understand the issue. The message itself is all right, only hMailServer responses to specific FETCH commands are wrong. This ticket here contains all the information you need.

Also, other clients might not have this issue depending what IMAP commands they use to get the headers/body of the forwarded message.