search

sendgrid / sendgrid-java

The Official Twilio SendGrid Led, Community Driven Java API Library
https://sendgrid.com
MIT License
485 stars 409 forks source link

Email is sent twice sometimes #420

Closed marpit closed 6 years ago

marpit commented 6 years ago

Email is sent twice sometimes. 

val from = new Email(fromAddress)
    val to = new Email(emailAddresses.mkString(";"))
    val mail = new Mail(from, subject, to, new Content("text/html", emailBody))
    mail.addPersonalization(getPersonalization(subject, emailAddresses))

    val request = new Request()

    request.setMethod(Method.POST)
    request.setEndpoint("mail/send")
    request.setBody(mail.build())

    sg.api(request)
thinkingserious commented 6 years ago

Hello @marpit,

To debug that, I suggest looking at the output of the to variable to see if the email is not included twice and the calling function to see if it's calling the above code twice.

With Best Regards,

Elmer

marpit commented 6 years ago

Hello @thinkingserious . Thank you for response, bat I assert size of input sequence and size corresponds with expected size. After I log sent emails (logging before send function) and result has expected size. After I check received emails and count of emails is higher that expected count.

thinkingserious commented 6 years ago

Hello @marpit,

By how much does the expected count differ?

Can you describe your logging methods? Perhaps there is an issue with the logging.

Also, I'd add some logging to capture my suggested debugging ideas.

With Best Regards,

Elmer

marpit commented 6 years ago

Hello @thinkingserious, Count is different about 2 emails ussualy (is received 14 emails instead of 12). So, I simplify the unit test and insert code and output there.

marpit commented 6 years ago 
test("SendEmails") {
    var i = 0;
    val toSend: Seq[SendGridEmail] = for( i <- 1 to 12)
      yield SendGridEmail(s"test: $i", Seq("example@mail.com"), s"This is email with index $i.")

    assert(toSend.size == 12)

    runAction[SendGridEmail](
      Seq(toSend),
      stream =>
        new EmailSenderSink(stream).declare(
          EmailSenderFactory,
          EmailSenderSink.Config(
            "SG.XXXXXXXX",
            "test@test.com")
      )
    )

Test send correct count of mails sometime, bat emails are duplicated very often. For example, received emails: https://pasteboard.co/H4ysKt6.png

And log:

08:38:41.117 [ScalaTest-run] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - Using manual clock
08:38:41.376 [ScalaTest-run] WARN  o.a.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
08:38:43.175 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] DEBUG c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - checkpointDir: C:\Users\H161558\AppData\Local\Temp\spark-29b512a4-fe98-4f57-b978-ec121823d6fc
08:38:43.562 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - Manual clock before advancing = 0
08:38:43.562 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - Manual clock after advancing = 1000
08:38:43.563 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.613 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.664 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.716 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.767 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.818 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.869 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.920 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.970 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.022 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.073 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.124 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.175 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.199 [Executor task launch worker-0] INFO  c.h.l.n.EmailSender - Email sent to List() with subject test: 1
08:38:44.199 [Executor task launch worker-1] INFO  c.h.l.n.EmailSender - Email sent to List() with subject test: 7
08:38:44.225 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.276 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.327 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.379 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.430 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.481 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.532 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.582 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.632 [Executor task launch worker-0] INFO  c.h.l.n.EmailSender - Email sent to List() with subject test: 2
08:38:44.632 [Executor task launch worker-1] INFO  c.h.l.n.EmailSender - Email sent to List() with subject test: 8
08:38:44.634 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.685 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.730 [Executor task launch worker-1] INFO  c.h.l.n.EmailSender - Email sent to List() with subject test: 9
08:38:44.731 [Executor task launch worker-0] INFO  c.h.l.n.EmailSender - Email sent to List() with subject test: 3
08:38:44.736 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.786 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.837 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.855 [Executor task launch worker-0] INFO  c.h.l.n.EmailSender - Email sent to List(m) with subject test: 4
08:38:44.855 [Executor task launch worker-1] INFO  c.h.l.n.EmailSender - Email sent to List() with subject test: 10
08:38:44.888 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.938 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.989 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.040 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.091 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.142 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.153 [Executor task launch worker-1] INFO  c.h.l.n.EmailSender - Email sent to List() with subject test: 11
08:38:45.154 [Executor task launch worker-0] INFO  c.h.l.n.EmailSender - Email sent to List() with subject test: 5
08:38:45.194 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.216 [Executor task launch worker-1] INFO  c.h.l.n.EmailSender - Email sent to List() with subject test: 12
08:38:45.216 [Executor task launch worker-0] INFO  c.h.l.n.EmailSender - Email sent to List() with subject test: 6
08:38:45.246 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.297 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.348 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.401 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO  c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - Output generated in 1838 milliseconds
thinkingserious commented 6 years ago

Hello @marpit,

Thanks for the update!

I have a couple points of feedback:

  1. Please do not post your SendGrid key publicly. For your account's safety I have removed it from your comment and we are going to deactivate that key. Please create a new one. I've also removed the emails in your log files to protect the privacy of that email.
  2. How did you install this SDK? I'm trying to determine what version you are using.
  3. I suggest you use different emails for the recipient to help further troubleshoot. For example (first.name+1@domain.com, first.name+2@domain.com, etc...)

Thanks!

With Best Regards,

Elmer

thinkingserious commented 6 years ago

Hello @marpit,

One more favor, could you please email us at dx@sendgrid.com with your SendGrid username?

Thank you!

With Best Regards,

Elmer

marpit commented 6 years ago
  1. SBT "com.sendgrid" % "sendgrid-java" % "4.1.2"

Username was sent to your email.

thinkingserious commented 6 years ago

Hi @marpit,

Could you please tell me the subject line of your email, as I'm unable to find it?

Thanks!

With Best Regards,

Elmer

marpit commented 6 years ago

Hello @thinkingserious,

Subject of email is "Sengrid username".

thinkingserious commented 6 years ago

Thank you!

One thing I might suggest is to not use the Mail constructor when sending multiple email, instead try this. I'm thinking the additional email may happen because you are using the constructor, plus adding a Personalization object.

With Best Regards,

Elmer

marpit commented 6 years ago

Hello @thinkingserious,

thank you for answer bat I don't see problem if I am using constructor. Bat I modified my test and now I am using constructor only without personalization object. 

def send(subject: String, emailAddresses: Seq[String], emailBody: String): Response = {
    logger.info(s"Email sent to $emailAddresses with subject $subject")

    val from = new Email(fromAddress)
    val to = new Email(emailAddresses.mkString(";"))
    val mail = new Mail(from, subject, to, new Content("text/html", emailBody))

    val request = new Request()

    request.setMethod(Method.POST)
    request.setEndpoint("mail/send")
    request.setBody(mail.build())

    sg.api(request)
  }

I don't see difference between our solution and your example code. I think that booth solutions will same results, same calls for our API. Can you explain where is difference please?

Thanks.

thinkingserious commented 6 years ago

Hi @marpit,

When you use the constructor, we create a personalization object for you behind the scenes. I was thinking that might be the cause of the extra email.

I was thinking that if you setup the personalization objects yourself, without the constructor, there would be less "magic" happening and we might figure out the root of the duplicate emails.

With Best Regards,

Elmer

sebash1992 commented 5 years ago

Hi, i am facing the same issue. is a Random behavior, not in all emails. I send every day 30 emails approximately. But sometimes one of those emails is received twice. I am using .net sendgrid library v9.11. The app is running as an azure function that runs 1 every day.

My code is

` public static async Task Send(string subject, string[] toUsers, string dataFromElastic, IConfigurationRoot config) { // Validate data coming in ValidateSendGridData(subject, toUsers);

        // Initialize SG client
        var client = new SendGridClient(config["SendGrid:Key"]);

        // Map to SG format
        var sendGridObject = SendGridMapper.MapToSendGridFormat(subject, toUsers, dataFromElastic, config);
        var sendGridObjectJson = JsonConvert.SerializeObject(sendGridObject);

        // Send request through SG API
        var response = await client.RequestAsync(method: SendGridClient.Method.POST,
                                                 requestBody: sendGridObjectJson,
                                                 urlPath: config["SendGrid:SendEndpoint"]);

        // It worked fine if it accepted the request
        // Generate response
        string responseBody = response.Body.ReadAsStringAsync().Result;

        SendGridResponse result = new SendGridResponse
        {
            StatusCode = (int)response.StatusCode,
            WasAccepted = response.StatusCode == System.Net.HttpStatusCode.Accepted,
            Errors = responseBody
        };

        return result;
    }

    private static void ValidateSendGridData(string subject, string[] toUsers)
    {
        if (string.IsNullOrEmpty(subject))
        {
            throw new Exception("Cannot send an email with an empty subject.");
        }

        if (toUsers.Length == 0)
        {
            throw new Exception("Cannot send an email with no to users");
        }
    }

public static SendGridRequestObject MapToSendGridFormat(string subject, string[] toUsers, string dataFromElastic, IConfigurationRoot config) { // TODO: we can add validations for the values received here

        // Create SG object used for the request
        SendGridRequestObject newObject = new SendGridRequestObject();
        Personalization[] personalizationArrayObj = new Personalization[1];

        From fromObject = new From
        {
            Email = config["SendGrid:FromEmail"],
            Name = config["SendGrid:FromName"]
        };

        int destinationUsersCount = toUsers.Length;
        To[] toArrayObj = new To[destinationUsersCount];

        for (int i = 0; i < destinationUsersCount; i++)
        {
            toArrayObj[i] = new To
            {
                Email = toUsers[i]
            };
        }

        // The subject has to travel inside of the dynamic template in order to work
        var dataFromElasticJSON = JObject.Parse(dataFromElastic);
        dataFromElasticJSON.Add("subject", subject);

        // Before creating the personalization, let's see if we have any users for bcc
        string bccUsersFromConfig = config["SendGrid:BccUsers"];
        Bcc[] bccArrayObj = null;

        if (!string.IsNullOrEmpty(bccUsersFromConfig))
        {
            // Add our users here
            // Remove empty strings after performing the split just in case
            string[] parsedBccUsers = bccUsersFromConfig.Split(Constants.SPLITTING_CHAR).Where(x => x.IsValidEmail()).ToArray();
            int bccUsersCount = parsedBccUsers.Length;

            bccArrayObj = new Bcc[bccUsersCount];

            for (int i = 0; i < bccUsersCount; i++)
            {
                bccArrayObj[i] = new Bcc
                {
                    Email = parsedBccUsers[i]
                };
            }
        }

        // Create personalization object
        Personalization persObj = new Personalization
        {
            To = toArrayObj,
            Bcc = bccArrayObj,
            DynamicTemplateData = JsonConvert.DeserializeObject(dataFromElasticJSON.ToString())
        };
        personalizationArrayObj[0] = persObj;

        newObject.From = fromObject;
        newObject.TemplateId = config["SendGrid:TemplateId"];
        newObject.Personalizations = personalizationArrayObj;

        // Return SG object ready to be sent
        return newObject;
    }
}

`