Replace the ([R][E][:]) regex for Message merging with a multi language Regex

[SimonZein]

Inside the message processing / message merging the Regex for an answer to an existing message is defined as ([R][E][:]) - This regex only takes english language into account. Instead of changing every line of code with additional languages, I defined a Regex for all known prefixes in line 922 and apply this regex in the message processing.

[SimonZein]

Hello Adam, i hope it is ok to create a new pull request, based on another branch (clone of my dev), The changes for the commentToAdd Fix is also applied here

[AdhocAdam]

i hope it is ok to create a new pull request, based on another branch (clone of my dev)

YUP! But did you add those other changes in your D2 branch or were they there when you branched? I only bring this up because it means this Pull Request has "all" of the changes. In which case this Pull Request effectively negates the other.

[SimonZein]

i hope it is ok to create a new pull request, based on another branch (clone of my dev)

YUP! But did you add those other changes in your D2 branch or were they there when you branched? I only bring this up because it means this Pull Request has "all" of the changes. In which case this Pull Request effectively negates the other.

I did that on purpose, as I didn't know if you want to implement that regex at all. And I have no experience with github, so I don't know what happens, when you merge both changes in, and the 2nd one doesn't have the changes of the first one 🙈🙈

[AdhocAdam]

Oh! Yeah keep them separate! Side note, both of these are great!

[AdhocAdam]

I clearly missed the "Update Branch" action on the Pull Request 🤦‍♂️ . Looks like everything is sorted now 😃

[AdhocAdam]

First thing first. This is...

Which reminded me of another issue I solved awhile back, wherein I discovered there was a method that handles illegal characters in file paths. That said, this has somewhat led me down a rabbit hole hoping to discover some kind of similar functionality for handling "RE:" in the subject of emails. So far I've only found a couple articles that highlight the proposed prefixes, but no formal MS docs that reference them.

In any case, I brought your changes/reply variable into PowerShell because I was curious how many more versions of this exist (14 - 😮). That said, this also reminded of #270. I'm pretty sure what your proposing does not conflict with that since the regex you laid out uses logical ORs. But now it makes me wonder how Outlook/Exchange handles this between senders of different locales (e.g. does it stack reply headers? does it only ever render one for the recipient? different across Exchange versions?)

In any case, I need to get a few things setup to test this properly. This has certainly piqued my curiosity.

Thank you again for raising this 😄

[SimonZein]

Hello Adam, thanks for the kind words :)

I have my List from here: https://en.wikipedia.org/wiki/List_of_email_subject_abbreviations

It also states, what happens, when a reply mail consists of an abbreviation in a different language:

[AdhocAdam]

...email clients may have an option...

All the more reason to test this out 😃

[AdhocAdam]

So I've tested this a few different ways (all in OWA using English, German, and Spanish):

1. Two users of different locales, both keeping SCSM on the thread and replying to everyone
2. Three users of different locales, both keeping SCSM on the thread and replying to everyone
3. Manually changing the RE: in the subject to something from your regex variable, in the middle of an existing thread that has an existing Work Item in SCSM

And in every case:

• the Exchange Conversation ID keeps the messages correctly threaded
• the connector respects every reply and adds each reply as a new Comment to the existing Work Item
• every sender sees their Reply header when they reply/send an email
  • English sends RE:, German sees the RE:, but German replies with AW:

I did have a question though on two of the regexes and I don't know if this was intentional on your part or not.

([R][E][:])
([A][W][:])
([S][V][:])
([A][n][t][w][:])
([V][S][:])
([R][E][F][:])
([V][á][:])
([R][I][F][:])
([S][V][:])
([B][L][S][:])
([A][t][b]\.[:])
([R][E][S][:])
([O][d][p][:])
([Y][N][T][:])
([A][T][B][:])

The Hungarian/Latvian reply prefixes of Vá: and Atb.: don't follow the syntax of the others in that they each have a character that isn't in [square brackets]. The thing is, in testing it seems to work fine with and without. But in any case, wanted to ask

[SimonZein]

Hello Adam, nice to hear it works as expected. Concerning the square brackets around every character: Tbh I never use them whenever I want to create a regex string out of normal characters. When validating the regex via regex101.com and also when testing, it just makes no difference - at least when I need them. Also in my Customevents I never use it that way. But as this is a pull request for the connector script I wanted to adapt to the way you do it :) And as I am not that used to regex as you are, I am pretty sure you have a good reason to use it that way. One step of creating regex out of strings for me is using the [regex]::escape(STRING) function in powershell and afterwards adding the square brackets to the characters. I guess in my function the á was not recognized as valid letter and therefor the square brackets have not been added around. So for your question : I also recognized it after the regex string was created but as it worked that way, I didn't change it. But it was not on purpose to exclude them ;)

[AdhocAdam]

One step of creating regex out of strings for me is using the

function in powershell

Welp. I learned something new today 😃

[SimonZein]

One step of creating regex out of strings for me is using the

function in powershell

Welp. I learned something new today 😃

I am honored of being able to show you something new ;)

[AdhocAdam]

Did some more testing around this, in short: everything looks great 👍. But to be a bit more verbose, I had several more exchanges back n forth between three different users in my lab of all different languages.

Scenario 1: [Work Item ID] is not the subject. In Outlook web, the replies are threaded correctly, but in Outlook desktop they don't get grouped correctly as a single threaded conversation. What's interesting here is examining the Conversation ID, they are all identical. The take away here is that this is fine, because it means when the connector goes to process emails that are missing the [Work Item ID] in the subject, Merge Replies would kick in and function as expected since they all share the same Conversation ID.

Scenario 2: [Work Item ID] is in the subject. In Outlook web, the replies are threaded correctly, but in Outlook desktop they don't get grouped correctly as a single threaded conversation. Examining the Conversation ID, they don't have one in common. But what they do have in common is the [Work Item ID] is in the subject, which means default connector behavior would be used and Merge Replies would not engage.

[SimonZein]

We have the reply regex running in productive and I have not found one issue with it so far. And we have subsidiaries around the world - so it is really helpful :)

[AdhocAdam]

Thank you for your contribution! This will ship with the next release.