search

salesagility / SuiteCRM

SuiteCRM - Open source CRM for the world
https://www.suitecrm.com
GNU Affero General Public License v3.0
4.41k stars 2.07k forks source link

Email Grouping/threading/conversation problem #7570

Open wolfkiss0110 opened 5 years ago

wolfkiss0110 commented 5 years ago

Issue

Email Threading/grouping problem. Email that sended does not group inside 1 conversation. I am using outlook.com email. Another email account like gmail and yahoo able to group but only outlook.com can not be group. If the email subject is add "RE:", then the gmail and yahoo can be group, but outlook.com will not be group.

Expected Behavior

All email sended should be grouping inside 1 conversation. Example: There are 2 case inside the system. [Server Down [CASE:1]] [Cashing Problem [CASE:2]]

[contact1 say "Down.."] [contact1 say "Crasing......."] [user1 say "Update ...."] [user1 say "Investigating.."]

The contact inbox: [SuiteCRM. 2 Server Down [CASE:1] ====Please reply...]
[SuiteCRM. 2 Crashing Problem [CASE:2] ====Please reply...]

Actual Behavior

All email sended are 1 conversation. Example: There are 2 case inside the system. [Server Down [CASE:1]] [Cashing Problem [CASE:2]]

[contact1 say "Down.."] [contact1 say "Crasing......."] [user1 say "Update ...."] [user1 say "Investigating.."]

The contact inbox: [SuiteCRM Server Down [CASE:1] ====Please reply...] [SuiteCRM Server Down [CASE:1] update ====Please reply...] [SuiteCRM Crashing Problem [CASE:2] ====Please reply...] [SuiteCRM Crashing Problem [CASE:2] update ====Please reply...]

Possible Fix

Add "In-Reply-To" header to the email before send. The "In-Reply-To" is the previous email message id.

For example: [SuiteCRM Server Down [CASE:1] ====Please reply...] <- after send this email save this message id of this email [SuiteCRM Server Down [CASE:1] update ====Please reply...] <- before send this email add custom header which is "In-Reply-To" in to this email

There are also a function in the PHPMailer which is getLastMessageID(). To retrieve last generate message id.

Steps to Reproduce

  1. contact send email to group inbound email account
  2. system create a new case and send case creation notification email to the contact
  3. user update the case
  4. system send contact case update notifcation email to the contact.
  5. Open contact inbox and that are 2 conversation inside the inbox.

Context

This is generate huge number email conversation in the inbox.

Your Environment

pgorod commented 5 years ago

Related information in this (still active) Forum thread: https://suitecrm.com/suitecrm/forum/suitecrm-7-0-discussion/26536-email-contact-case-update-email-doesn-t-send-in-the-same-email-thread

wolfkiss0110 commented 5 years ago

I have created some code to make it work. But I don't know if my code conflicts with another module or anything.

Step1: create a new function -> include\SugarPHPMailer.php 

public function getMessageID(){

        $message_id_0 = parent::getLastMessageID(); //get message id of last email
        $GLOBALS['log']->debug("getLastMessageID_0: ".$message_id_0);
        return $message_id_0;
    }

Step2: set custom header before send - > modules\AOP_Case_Updates\AOP_Case_Updates.php line 306

    $emailBean = BeanFactory::newBean('Emails');
    //get all email related to the case.
    $beanList = $emailBean->get_full_list('date_entered',"parent_id = '{$caseId}'");
    $refer = ""; //"references" header string
    $temp = ""; //"in-reply-to" header string
    foreach($beanList as $data){ //every single email related to the case.
        //check the email, it the email contain a message id with "@". The "@" is to determine the correct format of a message id.
        if(!empty($data->message_id) && strpos($data->message_id,"@") !== false){
            $temp = $data->message_id;//store latest message id
            $refer .= " <" . $temp . ">";//store whole flow of message id
        }
    }
    $mailer->addCustomHeader("References",$refer);//add "references" header
    $mailer->addCustomHeader("In-Reply-To","<".$temp.">");//add "in-reply-to" header

Step3: set custom header before send -> modules\Emails\Email.php line 3111 before send()

        // check the email is it related to cases module.
        if($this->parent_type == 'Cases'){
            $emailBean = BeanFactory::newBean('Emails');
            $beanList = $emailBean->get_full_list('date_entered',"parent_id = '{$this->parent_id}'");//get all email related to the case.
            $refer = "";//"references" header string
            $temp = "";//"in-reply-to" header string
            foreach($beanList as $data){//every single email related to the case.
                //check the email, it the email contain a message id with "@". The "@" is to determine the correct format of a message id.
                if(!empty($data->message_id) && strpos($data->message_id,"@") !== false){
                    $temp = $data->message_id;//store latest message id
                    $refer .= " <" . $temp . ">";//store whole flow of message id
                }
            }
            $mail->addCustomHeader("References",$refer);//add "references" header
            $mail->addCustomHeader("In-Reply-To","<".$temp.">");//add "in-reply-to" header
        }

Step4: store message id after send -> modules\Emails\Email.php line 3153 after END INBOUND EMAIL HANDLING

        if($this->parent_type == 'Cases'){
            $format = array("<",">");
            $this->message_id = str_replace($format,"",$mail->getMessageID());
        }

Step5: set custom header before send -> modules\AOP_Case_Updates\CaseUpdatesHook.php at sendClosureEmail function before send()

    $emailBean = BeanFactory::newBean('Emails');
    $beanList = $emailBean->get_full_list('date_entered',"parent_id = '{$case->id}'");
    $refer = "";
    $temp = "";
    foreach($beanList as $data){
        if(!empty($data->message_id) && strpos($data->message_id,"@") !== false){
            $temp = $data->message_id;
            $refer .= " <" . $temp . ">";
        }
    }
    $mailer->addCustomHeader("References",$refer);
    $mailer->addCustomHeader("In-Reply-To","<".$temp.">");

Step6: save message id -> modules\AOP_Case_Updates\CaseUpdatesHook.php at logEmail function before save()

$data = array("<",">");//format $emailObj->message_id = str_replace($data,"",$mailer->getMessageID());//store message id without "<" and ">"

wolfkiss0110 commented 5 years ago

Step2: User update from "Case Update Thread" image

Step3: User update from compose email image

Now left email send by workflow is not in same conversation. Like reminder email. image

wolfkiss0110 commented 5 years ago

image even though it already related to a case, but it still do not create a case update in the "Case Update Thread" after send the email.

Here is my code: modules\AOP_Case_Updates\CaseUpdatesHook.php -> saveEmailUpdate function public function saveEmailUpdate($email) {

    if ($email->intent !== 'createcase' || $email->parent_type !== 'Cases') {
        if($email->intent != 'createcase' && $email->parent_type == 'Cases' && $email->type == 'out' && $email->status=='sent' && $email->assigned_user_id != '1' && $email->mailbox_id != null){
            $GLOBALS['log']->debug("intent: {$email->intent}, parent_type: {$email->parent_type}, type: {$email->type}, status: {$email->status}");
        }else{
            $GLOBALS['log']->debug('CaseUpdatesHook: saveEmailUpdate: Not a create case or wrong parent type');
            return;
        }
    }

    if (!isAOPEnabled()) {
        $GLOBALS['log']->debug('CaseUpdatesHook: false AOPEnabled');
        return;
    }

    if (!$email->parent_id) {
        $GLOBALS['log']->debug('CaseUpdatesHook: saveEmailUpdate No parent id');

        return;
    }

    if ($email->cases) {
        if($email->intent != 'createcase' && $email->parent_type == 'Cases' && $email->type == 'out' && $email->status=='sent' && $email->assigned_user_id != '1' && $email->mailbox_id != null){
            $GLOBALS['log']->debug("intent: {$email->intent}, parent_type: {$email->parent_type}, type: {$email->type}, status: {$email->status}");
        }else{
            $GLOBALS['log']->debug('CaseUpdatesHook: saveEmailUpdate cases already set');

            return;
        }
    }

    if ($email->fetched_row['parent_id']) {
        $GLOBALS['log']->debug('CaseUpdatesHook: Will have been processed already');
        //Will have been processed already
        return;
    }

    $updateText = $this->unquoteEmail($email->description_html ? $email->description_html : $email->description);

    if(!isset($updateText)){
        $GLOBALS['log']->debug('CaseUpdatesHook: description empty');
        return;
    }

    $caseUpdate = new AOP_Case_Updates();

    $caseUpdate->case_id = $email->parent_id;
    $caseUpdate->description = $updateText;
    $caseUpdate->internal = false;

    if($email->intent != 'createcase' && $email->parent_type == 'Cases' && $email->type == 'out' && $email->status=='sent' && $email->assigned_user_id != '1'){
        $caseUpdate->assigned_user_id = $email->assigned_user_id;
        $caseUpdate->internal = true; //prevent sending 2 email.(compose email and contact case update email)
    }else{
        $ea = new SugarEmailAddress();
        $beans = $ea->getBeansByEmailAddress($email->from_addr);
        $contact_id = null;
        foreach ($beans as $emailBean) {
            if ($emailBean->module_name === 'Contacts' && !empty($emailBean->id)) {
                $contact_id = $emailBean->id;
                $this->linkAccountAndCase($email->parent_id, $emailBean->account_id);
            }
        }
        $caseUpdate->contact_id = $contact_id;
        $this->updateCaseStatus($caseUpdate->case_id);
    }

    $caseUpdate->name = $email->name;
    $caseUpdate->save();
    $notes = $email->get_linked_beans('notes', 'Notes');
    foreach ($notes as $note) {
        //Link notes to case update also
        $newNote = BeanFactory::newBean('Notes');
        $newNote->name = $note->name;
        $newNote->file_mime_type = $note->file_mime_type;
        $newNote->filename = $note->filename;
        $newNote->parent_type = 'AOP_Case_Updates';
        $newNote->parent_id = $caseUpdate->id;
        $newNote->save();
        $srcFile = "upload://{$note->id}";
        $destFile = "upload://{$newNote->id}";
        copy($srcFile, $destFile);
    }
}
wolfkiss0110 commented 5 years ago

for email send from workflow modules\AOW_Actions\actions\actionSendEmail.php at line 518

if($relatedBean->module_dir == 'Cases'){
            $emailBean = BeanFactory::newBean('Emails');
            //get all email related to the case.
            $beanList = $emailBean->get_full_list('date_entered',"parent_id = '{$relatedBean->id}'");
            $refer = ""; //"references" header string
            $temp = ""; //"in-reply-to" header string
            foreach($beanList as $data){ //every single email related to the case.
                //check the email, it the email contain a message id with "@". The "@" is to determine the correct format of a message id.
                if(!empty($data->message_id) && strpos($data->message_id,"@") !== false){
                    $temp = $data->message_id;//store latest message id
                    $refer .= " <" . $temp . ">";//store whole flow of message id
                }
            }
            $mail->addCustomHeader("References",$refer);//add "references" header
            $mail->addCustomHeader("In-Reply-To","<".$temp.">");//add "in-reply-to" header
        }

before save $emailObj

            $format = array("<",">");
            $emailObj->message_id = str_replace($format,"",$mail->getMessageID());
RickfiregamesPT commented 2 weeks ago

Hey Guys, is there an update about this issue? It has been so long so i hope you where able to find a solution @wolfkiss0110 If so can you share it with me please? Facing the same problem here :D

chris001 commented 2 weeks ago

@wolfkiss0110 Would you make a Pull Request with your code, and click to sign online the Contributor Licensing Agreement (CLA)?