Closed marpit closed 6 years ago
Hello @marpit,
To debug that, I suggest looking at the output of the to
variable to see if the email is not included twice and the calling function to see if it's calling the above code twice.
With Best Regards,
Elmer
Hello @thinkingserious . Thank you for response, bat I assert size of input sequence and size corresponds with expected size. After I log sent emails (logging before send function) and result has expected size. After I check received emails and count of emails is higher that expected count.
Hello @marpit,
By how much does the expected count differ?
Can you describe your logging methods? Perhaps there is an issue with the logging.
Also, I'd add some logging to capture my suggested debugging ideas.
With Best Regards,
Elmer
Hello @thinkingserious, Count is different about 2 emails ussualy (is received 14 emails instead of 12). So, I simplify the unit test and insert code and output there.
test("SendEmails") {
var i = 0;
val toSend: Seq[SendGridEmail] = for( i <- 1 to 12)
yield SendGridEmail(s"test: $i", Seq("example@mail.com"), s"This is email with index $i.")
assert(toSend.size == 12)
runAction[SendGridEmail](
Seq(toSend),
stream =>
new EmailSenderSink(stream).declare(
EmailSenderFactory,
EmailSenderSink.Config(
"SG.XXXXXXXX",
"test@test.com")
)
)
Test send correct count of mails sometime, bat emails are duplicated very often. For example, received emails: https://pasteboard.co/H4ysKt6.png
And log:
08:38:41.117 [ScalaTest-run] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - Using manual clock
08:38:41.376 [ScalaTest-run] WARN o.a.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
08:38:43.175 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] DEBUG c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - checkpointDir: C:\Users\H161558\AppData\Local\Temp\spark-29b512a4-fe98-4f57-b978-ec121823d6fc
08:38:43.562 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - Manual clock before advancing = 0
08:38:43.562 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - Manual clock after advancing = 1000
08:38:43.563 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.613 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.664 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.716 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.767 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.818 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.869 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.920 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:43.970 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.022 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.073 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.124 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.175 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.199 [Executor task launch worker-0] INFO c.h.l.n.EmailSender - Email sent to List() with subject test: 1
08:38:44.199 [Executor task launch worker-1] INFO c.h.l.n.EmailSender - Email sent to List() with subject test: 7
08:38:44.225 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.276 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.327 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.379 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.430 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.481 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.532 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.582 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.632 [Executor task launch worker-0] INFO c.h.l.n.EmailSender - Email sent to List() with subject test: 2
08:38:44.632 [Executor task launch worker-1] INFO c.h.l.n.EmailSender - Email sent to List() with subject test: 8
08:38:44.634 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.685 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.730 [Executor task launch worker-1] INFO c.h.l.n.EmailSender - Email sent to List() with subject test: 9
08:38:44.731 [Executor task launch worker-0] INFO c.h.l.n.EmailSender - Email sent to List() with subject test: 3
08:38:44.736 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.786 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.837 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.855 [Executor task launch worker-0] INFO c.h.l.n.EmailSender - Email sent to List(m) with subject test: 4
08:38:44.855 [Executor task launch worker-1] INFO c.h.l.n.EmailSender - Email sent to List() with subject test: 10
08:38:44.888 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.938 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:44.989 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.040 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.091 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.142 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.153 [Executor task launch worker-1] INFO c.h.l.n.EmailSender - Email sent to List() with subject test: 11
08:38:45.154 [Executor task launch worker-0] INFO c.h.l.n.EmailSender - Email sent to List() with subject test: 5
08:38:45.194 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.216 [Executor task launch worker-1] INFO c.h.l.n.EmailSender - Email sent to List() with subject test: 12
08:38:45.216 [Executor task launch worker-0] INFO c.h.l.n.EmailSender - Email sent to List() with subject test: 6
08:38:45.246 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.297 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.348 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - batches run = 0, numBatches = 1
08:38:45.401 [ScalaTest-run-running-OrgSettingAlertTransformLocalizeSendMailTest] INFO c.h.l.n.s.t.OrgSettingAlertTransformLocalizeSendMailTest - Output generated in 1838 milliseconds
Hello @marpit,
Thanks for the update!
I have a couple points of feedback:
Thanks!
With Best Regards,
Elmer
Hello @marpit,
One more favor, could you please email us at dx@sendgrid.com with your SendGrid username?
Thank you!
With Best Regards,
Elmer
"com.sendgrid" % "sendgrid-java" % "4.1.2"
Username was sent to your email.
Hi @marpit,
Could you please tell me the subject line of your email, as I'm unable to find it?
Thanks!
With Best Regards,
Elmer
Hello @thinkingserious,
Subject of email is "Sengrid username".
Thank you!
One thing I might suggest is to not use the Mail
constructor when sending multiple email, instead try this. I'm thinking the additional email may happen because you are using the constructor, plus adding a Personalization
object.
With Best Regards,
Elmer
Hello @thinkingserious,
thank you for answer bat I don't see problem if I am using constructor. Bat I modified my test and now I am using constructor only without personalization object.
def send(subject: String, emailAddresses: Seq[String], emailBody: String): Response = {
logger.info(s"Email sent to $emailAddresses with subject $subject")
val from = new Email(fromAddress)
val to = new Email(emailAddresses.mkString(";"))
val mail = new Mail(from, subject, to, new Content("text/html", emailBody))
val request = new Request()
request.setMethod(Method.POST)
request.setEndpoint("mail/send")
request.setBody(mail.build())
sg.api(request)
}
I don't see difference between our solution and your example code. I think that booth solutions will same results, same calls for our API. Can you explain where is difference please?
Thanks.
Hi @marpit,
When you use the constructor, we create a personalization object for you behind the scenes. I was thinking that might be the cause of the extra email.
I was thinking that if you setup the personalization objects yourself, without the constructor, there would be less "magic" happening and we might figure out the root of the duplicate emails.
With Best Regards,
Elmer
Hi, i am facing the same issue. is a Random behavior, not in all emails. I send every day 30 emails approximately. But sometimes one of those emails is received twice. I am using .net sendgrid library v9.11. The app is running as an azure function that runs 1 every day.
My code is
` public static async Task
// Initialize SG client
var client = new SendGridClient(config["SendGrid:Key"]);
// Map to SG format
var sendGridObject = SendGridMapper.MapToSendGridFormat(subject, toUsers, dataFromElastic, config);
var sendGridObjectJson = JsonConvert.SerializeObject(sendGridObject);
// Send request through SG API
var response = await client.RequestAsync(method: SendGridClient.Method.POST,
requestBody: sendGridObjectJson,
urlPath: config["SendGrid:SendEndpoint"]);
// It worked fine if it accepted the request
// Generate response
string responseBody = response.Body.ReadAsStringAsync().Result;
SendGridResponse result = new SendGridResponse
{
StatusCode = (int)response.StatusCode,
WasAccepted = response.StatusCode == System.Net.HttpStatusCode.Accepted,
Errors = responseBody
};
return result;
}
private static void ValidateSendGridData(string subject, string[] toUsers)
{
if (string.IsNullOrEmpty(subject))
{
throw new Exception("Cannot send an email with an empty subject.");
}
if (toUsers.Length == 0)
{
throw new Exception("Cannot send an email with no to users");
}
}
public static SendGridRequestObject MapToSendGridFormat(string subject, string[] toUsers, string dataFromElastic, IConfigurationRoot config) { // TODO: we can add validations for the values received here
// Create SG object used for the request
SendGridRequestObject newObject = new SendGridRequestObject();
Personalization[] personalizationArrayObj = new Personalization[1];
From fromObject = new From
{
Email = config["SendGrid:FromEmail"],
Name = config["SendGrid:FromName"]
};
int destinationUsersCount = toUsers.Length;
To[] toArrayObj = new To[destinationUsersCount];
for (int i = 0; i < destinationUsersCount; i++)
{
toArrayObj[i] = new To
{
Email = toUsers[i]
};
}
// The subject has to travel inside of the dynamic template in order to work
var dataFromElasticJSON = JObject.Parse(dataFromElastic);
dataFromElasticJSON.Add("subject", subject);
// Before creating the personalization, let's see if we have any users for bcc
string bccUsersFromConfig = config["SendGrid:BccUsers"];
Bcc[] bccArrayObj = null;
if (!string.IsNullOrEmpty(bccUsersFromConfig))
{
// Add our users here
// Remove empty strings after performing the split just in case
string[] parsedBccUsers = bccUsersFromConfig.Split(Constants.SPLITTING_CHAR).Where(x => x.IsValidEmail()).ToArray();
int bccUsersCount = parsedBccUsers.Length;
bccArrayObj = new Bcc[bccUsersCount];
for (int i = 0; i < bccUsersCount; i++)
{
bccArrayObj[i] = new Bcc
{
Email = parsedBccUsers[i]
};
}
}
// Create personalization object
Personalization persObj = new Personalization
{
To = toArrayObj,
Bcc = bccArrayObj,
DynamicTemplateData = JsonConvert.DeserializeObject(dataFromElasticJSON.ToString())
};
personalizationArrayObj[0] = persObj;
newObject.From = fromObject;
newObject.TemplateId = config["SendGrid:TemplateId"];
newObject.Personalizations = personalizationArrayObj;
// Return SG object ready to be sent
return newObject;
}
}
`
Email is sent twice sometimes.