Emails sent via IMAP received once and shows only once on recipient's Inbox, but email shows twice on Sent Items

[joshscorpion]

Issue

The issue which is hard to explain unless seen, is that the recipient gets x1 copy of the email intended, but the sender will have x2 emails on the Sent Items.

I couldn't understand how this is possible, having x2 items on our Sent Items and only x1 copy on the Recipient's email box. Looked through all the boxes for the account, i.e. spam, deleted, etc., but only x1 email is indeed received.

A colleague then compared the two emails and found there is something about "content-language", where the email that shares most of the info is when "content-language=en-US", the other email shows as "pt-BR".

Not sure where the "pt-BR" comes from, either than our initial domain is registered in Brazil, our owners are Brazilian, but all the emails are handled by our agreement with Microsoft and Office 365, through Outlook.

Images above share the duplicate emails. I dont understand also where the "content-language" comes from, we haven't added "content-language" to anything; additionally for the email purpose we only want to use "en-US".

Expected Behavior

What should happen is that if we send x1 email, the Sent Items should only show x1 email, else after the first Campaign the confusion would be overwhelming.

Actual Behavior

If I send an email, the recipient gets x1 email as he/she should. Whatever account of ours that sends that email will have x2 emails o their Sent Items; as shared on the images above.

Possible Fix

I have already: Went to the config_override and added '''$sugar_config['default_email_charset'] = 'US-ASCII';''', only after doing it that I could setup the initial logins for the users accounts.

Context

This is a really serious bug for us, we are a small startup and need to start using the CRM to better manage our time, start with Campaigns, email templates, pdf templates, etc., being a small team being transparent and productive is important, the email component will help us a lot; else we'll be forced to use Outlook separately and there we dont have the templates, etc.

One link that greatly helped was github.com/salesagility/SuiteCRM/issues/6866, it gave us solutions that worked with Exchange and the connection.

Our Environment

• Ubuntu 18.04
• SuiteCRM 7.11.18
• SuiteCRM has x2 languages, "en-US" and "pt-PT" enabled and working, main language "en-US"
• PHP 7.2
• Exchange via Office 365
• Same problem using Gmail accounts

[johnM2401]

Hey there,

I'm currently not able to replicate, I'm afraid Could you perhaps provide a little more info?

Such as:

• Does this issue occur when sending emails FROM SuiteCRM? (if so, is it via Email->Compose , Campaigns or Elsewhere?) Or are the Emails being sent via an External Mail client, and being imported into the CRM?

• Where are you viewing these duplicate "Sent" emails? Is this via the "Sent" folder of the Mailbox in the Email Module, or via the related Record's Subpanels, or elsewhere?

• And, finally, are there any odd steps that are needed to replicate? Or, is this a consistent issue, found with each Email Send? (ie, even for users in the pt-PT language? Or users sending from Brazil, if any?)

Thanks!

[joshscorpion]

Hey johnm2401,

Really happy to hear back from you.

Below I'll answer each of the questions you raised:

1. Does this issue occur when sending emails FROM SuiteCRM? On Campaigns sends only once, on Compose shows x2 emails on Sent Items. Not using external email client, no imports of emails either.

2. Where are you viewing these duplicate "Sent" emails? If we go to Sent Items either on SuiteCRM, Outlook, tested it with Gmail and on Compose shows x2 emails on the Sent Items for that email account.

We can see the duplicates on any device that has access to the email account and the Sent Items for that email.

3. And, finally, are there any odd steps that are needed to replicate? As long as we send an email, except with Campaigns, it will always show x2 emails on the Sent Items and x1 email on the Inbox for the recipient.

Its so strange I cant explain it better, I always thought that if an email shows in our SMTP, on success x1 email will end up in the recipient box, how can x2 emails be shows and only x1 email is baffling.

The major difference is that the email that is correct has content-language=en-US, the 2nd email internally has little information either than content-language=pt-BR.

The pt-BR part baffles us even more, because I am the one doing the tests, our server is US based, we have no systems in pt-BR, added 2nd language on SuiteCRM, pt-PT recently but current focus is having our CRM working in English first, then complete the job for the Portuguese users.

[From suitecrmlog, is this when its sending twice?]

I am the one that thanks for your time. Let me know how I can help more.

[johnM2401]

Hey,

Thank you for the additional info!

I've been able to replicate it with this, so I've marked it as a bug.

(Using Gmail, for reference)

[joshscorpion]

Hey johnM2401,

Fantastic news, driving me nuts over here.

I'll appreciate if you hear of any news/update on this bug, truly appreciate your time and expertise.

Simply fantastic!

[joshscorpion]

Hey johnM2401,

Fantastic news, driving me nuts over here.

I'll appreciate if you hear of any news/update on this bug, truly appreciate your time and expertise.

Simply fantastic!

[joshscorpion]

Hi johnM2401,

Some additional information about the duplication of "things" that is happening, also with other items, its not only x1 bug, I have x3-x4 of those bugs where it duplicates things, not only with emails that are being shown twice on Sent Items; at least the recipient only gets one copy, could be worse.

Other actions that are also happening x2 instead of only x1 time.

** When selecting an Account/Contact/Lead/etc as a favorite, it adds it twice on the left sidebar

** When clicking on any hyperlink, it would open x2 windows, instead of one window I refer to it on the image above, for some reason today it only opens x1 window, don't know why.

** On Scheduler, all events are triggering twice, i.e. AOD Index runs twice at the same date/hour Below two examples of two Schedulers

Some important updates about our systems.

We were running php7.2, planned to migrate from php7.2 to php7.3 (for performance and compatibility issues) as we are using SuiteCRM 7.11.18 and 7.11.18 doesn't support php7.4 or php8.0. On the server (CLI) we also wanted to migrate from php7.2 to php7.3.

Two weeks ago there was a Linux update and another one this past week. I wrote a script that does updates removes autoremoves autoclean clean, saw the updates, showed no error, I thought all was well; it wasn't.

By mid week when I went to the Schedulers, all had stopped, I couldn't believe it, this was quite a serious bug as the Scheduler handles so many crucial items such as the various types of indexing, processing of the Workflows, etc. Without the Workflows many business operations would be seriously affected.

For us especially the Workflows will be more and more important. We are a small team and need to automate as much as possible when it comes to communication and marketing. Having the Workflows stopped meant the business rules that were being agreed could not be tested, delivered.

It took me 3 days and about 5 hours sleep to get the scheduling and the indexing working again.

The problem was that on one of the latest Linux updates, it installed php8.0 as CLI, the php7.2 (client-side) carried on being operational; when opening my phpinfo.php it would show php7.2, only on the server that if I did a "php -v" that it showed me the server had php8.0 as the CLI.

Going through the logs I saw errors that there were php/javascript errors on libraries that I had never seen or even knew what they were. As I am not a developer I shared the errors with a "real" developer and he too could not see the errors and the syntax on the code was correct. That told me there was a php version issue.

So I installed php7.3 on the server, totally removed php8.0 from the server, the CLI is also php7.3 now, reconfigured php7.3 php.ini file with the settings I knew we need for our operation.

After that the Scheduler started firing x2 times for all Events that it handles, but at least it works and is indexing. This weekend I am testing the Workflows, as I shared above as the Scheduler stopped working, the Workflows had stopped working since last week, I am hoping this too now only triggers once; will update here.

** MySQL 8 This week managed to migrate our DBs from MySQL 5.7 to MySQL 8, v8 is about 20-25% faster, but it consumes a lot more resources than v5.7.

I ended up taking about 2 days, about 30+ hours constantly re-configuring MySQL, re-tuning it to lower resource usage.

Was a little frustrating as MySQL is the 1st MySQL version that comes with no mysql.cnf; it indeed creates one but is totally clean, apart from loading two other libraries. Through 2 days with almost no sleep managed to add quite a number of entries that helped me control MySQL 8 to some level, else our server with 4GB RAM could not handle it, even though it is faster than 5.7

MySQL 8 also doesn't support query caching, a bit of a pity, I understand its architecture changed but with MySQL 8 being so resource hungry query caching would help.

** Apache With MySQL 8 being so resource hungry, through the apache2.conf lowered available resources, i.e. MaxClients, ServerLimits, etc., this has helped a lot as we don't have that many users and needed additional resources to be freed by Apache to allow for more RAM to be available to MySQL 8.

** Swap I installed a 2GB swap and made it active, can already see its impact, one example when I run "free -m".

Hope all the info above will further help find this "bug" that duplicates efforts, truly strange.

Many thanks!

[johnM2401]

Hey,

Thank you for all the additional information. I'm sure it'll be of assistance if/when someone investigates these issues.

Unfortunately, I've never experienced either of the issues:

• 2x Windows when clicking a Hyperlink
• Double Scheduled Job runs.

For the latter, I assume it's not as simple as there just being two identical cron jobs running on the server, for this instance?

Either way, if you are able to reliably replicate either of these issues, then please raise it as a new issue. (This will make it much easier to track in the long-run)

Thanks again!

[joshscorpion]

Hey,

Quick update about the x2 scheduling issue that was happening.

On the cron already had done the usual as set by SuiteCRM

sudo crontab -e -u www-data ... added to the crontab file as per SuiteCRM recommendations, normal stuff:

• • • • • cd /var/www/example.com; php -f cron.php > /dev/null 2>&1

Found a second cron entry running on /etc/crontab:

• • • • • www-data cd /var/www/example.com; php -f cron.php > /dev/null 2>&1

Couldn't understand how this entry was present as I am the only person that admins the server; nevertheless commented the line out of that crontab and now the Schedulers are all triggering x1 as they should.

Tested to send an email, hoped it would deal with the x2 emails on Sent Items, the Recipient still gets only x1 email and Sent Items has x2 emails.

Tested all to add an Account/Contact/etc. as a Favorite, they all still add the favorite x2 on the left sidebar; instead of adding a Favorite x1 once as it should.

Many thanks for your time and expertise, they are appreciated.