email address validation and email processing

[vohmar]

after dropping postal addresses the importance of the remaining contact data incl email address is that much higher.

email address validations:

1. entered email address is correctly formed as email address
2. domain is registered and in zone (internal check for local zones, dns query for the domain part of email address for other TLDs)
3. there is an email server answering on that domain (dns query for mx record and ping?)

on contact:create and contact:update for email address all the validations should be run

on domain:create, domain:udpate, domain:renew and before sending any emails internal domain registration verification must be performed

all emails sent by the registry must go out using Que email sending que jobs must be configured so that in case of fatal error - ie nxdomain , malformed e-mail address the error is logged and the que job does not get re-run. In case of potentially temporary errors - mailserver not reploying, mailbox rejecting the email there must be a limit on how many times or for how long does the registry retry to send the email ie up to 48 hours.

[artur-intech]

Check if "validates_email_format_of" gem can be removed. Email can be first validated just with /@/ pattern and then using some DNS

• https://stackoverflow.com/questions/156430/regexp-recognition-of-email-address-hard
• https://softwareengineering.stackexchange.com/questions/78353/how-far-should-one-take-e-mail-address-validation

[teadur]

Whats the problem with validates_email_format_of ? It seems to include format validation & MX check, both what would be useful to us, why you want to rewrite that functionality ?

I would extend the issue with logic that if email isnt reachable it would be reflecetd in contact object, so broken addressess can be found from registry and information about them can be sent to registrars.

[artur-intech]

As I see from "Commits" and "Issue" sections, that gem is not maintained anymore. This is an additional dependency we need to maintain, so there should be unbeatable advantage over custom solution.

So:

• Format validation is done with simple /@/ regexp
• MX check can probably be done without using gem

P.S. What I wanted first is to check if we can get rid of that gem, not to rewrite it.

[teadur]

I would disagree that simple /@/ regexp is enough for email address validation.

[artur-intech]

https://haacked.com/archive/2007/08/21/i-knew-how-to-validate-an-email-address-until-i.aspx/

[yulgolem]

Got this kind of a gem, maintained well - https://github.com/rubygarage/truemail

It can validate: 1) If email address is correct by customizable regexp. 2) If email domain is registered by DNS and if it got MX records 3) If email server accepts mails for this email.

[vohmar]

• all attempts to create a new contact result in empty response - internal error on the portal side (env in production settings)

  <?xml version="1.0" encoding="UTF-8"?>
  <epp xmlns="https://epp.tld.ee/schema/epp-ee-1.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="lib/schemas/epp-ee-1.0.xsd">
  <response>
  <trID>
    <clTRID>timo-1594307526</clTRID>
    <svTRID>ccReg-0169201076</svTRID>
  </trID>
  </response>
  </epp>

[teadur]

Got this kind of a gem, maintained well - https://github.com/rubygarage/truemail

It can validate:

1. If email address is correct by customizable regexp.
2. If email domain is registered by DNS and if it got MX records
3. If email server accepts mails for this email.

All this is done before sending every email ? If for some reason for example DNS fails at first try what's going to happen ?

[yulgolem]

Well, the validation will fall and the email will not be saved. That's the MX check flow - https://github.com/rubygarage/truemail#rfc-mx-lookup-flow .

The validation layers of this gem are

[Whitelist/Blacklist] -> [Regex validation] -> [MX validation] -> [SMTP validation]

So each next layer includes previous validations - then we set validation level to :mx, gem checks lists, regex and MX.

[yulgolem]

UPDATE: No, not before sending emails - before the contact creation (could block creation of contacts with non-valid emails) and then once per month or so, just for info purposes.

When we are sending emails our email server makes something like mx/smtp checking by itself to find recipient's email server and we have no need to double that.