Bad handling/splitting of e-mail replies

[sinister3vil]

Hello and happy new year,

I'm revisiting a constant pain with Freescout. It's quite common for us to have communications made between other ticketing systems, not e-mail clients, which probably handle HTML in e-mails in a non-standard manner leading to issues when Freescout in parsing replies, in most cases incorrectly using the quoted text/signature part rather than the actual reply part, as the new reply text.

Related code https://github.com/freescout-helpdesk/freescout/blob/dist/app/Console/Commands/FetchEmails.php#L1287C9-L1327C10 Separators https://github.com/freescout-helpdesk/freescout/blob/1816aa276172cc3527fb0910a3376adb4a5ff5bb/app/Misc/Mail.php#L54C5-L73C7

Actual HTML examples are a bit hard to come by and, in any case, irrelevant as trying to support all different combinations is probably impossible.

Would any of the below workarounds be possible to implement in a future version :

1. Add an option to not split e-mails and use the full body as the Freescout message content. When enabled, whatever you get in the e-mail, regardless of how many levels of quoted replies it might have, will be added to the Freescout message.
2. Add a field to enter custom regex matches for the $alternative_reply_separators array.
3. Keep current splitting behavior but also keep the original message in a separate field. A user can then evaluate if this is a case of bad splitting, view the original field and edit the message replacing the text. This is what I, as a senior engineer with access to both Freescout and the actual e-mail inbox, am doing right now, manually. Note that "show original" right now does not show the original message as retrieved from the e-mail inbox, rather it shows the split part but before any html parsing or sanitation is made by Freescout.

Code-wise, 1. seems easy to implement by exposing a toggle button under settings and checking for that in the if clause. Maybe add a popup when toggling the option on that warns that "no parsing will be made and messages might get huge".

PHP version: PHP 7.4.33 FreeScout version: 1.8.115 Database: Mysql (5.5.5-10.1.48-MariaDB-0ubuntu0.18.04.1) Are you using CloudFlare: No

[freescout-helpdesk]

You can try to implement these things via a custom module first.

Also try to figure out why the original message can't be loaded from your mail server. Maybe you deleting original emails from the mail server.

[freescout-helpdesk]

We've just added an alternative reply separation method in the master branch, it will be published in the next release.

In order to enable it set APP_ALTERNATIVE_REPLY_SEPARATION=true parameter in the .env file and clear cache.

[sinister3vil]

Wow, thanks for the quick fix. We'll be sure to test this and revert on it's behavior.

Without wanting to sound like a wise-ass, just wanted to point out that the method is misspelled getHashedRelySeparator, instead of getHashedReplySeparator.