Bad email address in bulk emailing

[hgani]

Hi,

I am not sure if this is more of an issue on the gem side or on Mandrill's mail server side.

I tried sending to multiple recipients at once and sometimes it fails with the following exception:

Error Class Net::SMTPServerBusy Error Message 401 4.1.3 Bad recipient address syntax

In the past this has happened when I have email addresses with bad format, e.g. test@test..com (double dots). Note that I've done regex validation but some addresses just slip past my validation -- that's the reality of regex email validation (See http://stackoverflow.com/questions/201323/using-a-regular-expression-to-validate-an-email-address/719543#719543)

When sending emails in bulk, it's not possible for me to find out which email address is bad because I am sending to 1000 recipients in one go.

The sending fails for that whole batch so that even though I have an auto-retry solution (using sidekiq), this batch always keeps failing.

Ideally the gem or Mandril should tell me which of the email addresses is invalid (may be via webhook or using a more explicit exception that can be rescued and processed), so that I can blacklist the offending address. So even though the sending fails the first time, it will work on the next retry because by that time I've blacklisted the bad address.

Thanks

[tardate]

@hgani the mandrill-rails gem is not involved in mail sending .. that's a conversation between your stack and the SMTP gateway. NB: tho you can use mandrill-rails to receive inbound mail and webhook callbacks from mandrill related to mail delivery. So this is not really the place to find a good solution .. hence why I'm closing the issue, sorry!

That being said, yes, invalid addresses often cause the SMTP gateway to immediately reject the mail even before it gets into the queue (and not give you much information in the process).

I've tried various regex/gem solutions for email address validation, but none are perfect. Rather than try and re-invent the wheel, I've found the most reliable approach is to leverage some stuff lurking in ActionMailer (assuming you are sending from Rails). There isn't an obvious interface for this (I don't think it was an anticipated use-case), but here's one trick:

  def invalid_email?(email)
    Mail::ToField.new email
    raise Mail::Field::ParseError.new('Mail::AddressListsParser', email, "email is not fully qualified") unless email =~ /@/
    false
  rescue Mail::Field::ParseError => e
    "(#{e.message})"
  end

invalid_email? "test@test.com"
 => false 
invalid_email? "test@test..com"
 => "(Mail::AddressListsParser can not parse |test@test..com|\nReason was: Expected one of !, #, $, %, &, ', *, +, -, /, =, ?, ^, _, `, {, |, }, ~, \r\n, (, \", ., @, :, < at line 1, column 15 (byte 15) after .com)" 
invalid_email? "test"
 => "(Mail::AddressListsParser can not parse |test|\nReason was: email is not fully qualified)" 

Note: the reason the custom raise Mail::Field::ParseError is to also handle cases where email does not have an @domain qualifier (Mail::ToField.new "test" is considered "valid")

[hgani]

@tardate Thanks for the quick response and detailed explanation.

In the end, I implemented something very similar to your suggested solution

But I decided to use a gem (https://github.com/validates-email-format-of/validates_email_format_of) because the gem claims to adhere to RFC 2822 and RFC 3696 which should be compatible with Mandrill (on the other hand, Mail::ToField doesn't seem to be well documented so perhaps it's not meant to be used publicly)

def self.email_invalid?(email)
  ValidatesEmailFormatOf::validate_email_format(email).present?
end