From email-adress / domain adress confusion

[magnolia61]

First of all: great extension. This sort of stuff should find it's way into core I believe.

We have a small problem with the extension though: We have our website at ourdomain.nl but send email as ourdomainandsomething.nl (with spf and all in place. We have configured the from:email-adress as info@ourdomainandsomething.nl

But when enabled the ode-extension thinks we should have a info@ourdomain.nl adress. E.g. It seems it doesn't seem to look at the configured from-adress but at the adress the website is located and accessed at. Must be a glitch of some sort, right?

[JoeMurray]

Thanks.

Can you confirm that From Email Address configured at Administer > Communications > Organization Address and Contact Info is info@ourdomainandsomething.nl ? There are several places where one configures domain emails - this is the one used by the extension.

If not, could you provide us with CMS version, CiviCRM version, and ODE extension version?

[magnolia61]

at /civicrm/admin/domain?action=update&reset=1 we have info@ourdomainandsomething.nl at /civicrm/admin/options/from_email_address?reset=1 we have info@ourdomainandsomething.nl /civicrm/admin/options/from_email_address?reset=1 we have "Name of our org" info@ourdomainandsomething.nl

We use drupal 7.41 / CiviCRM 4.6.9 & 4.7.1 / and ODE 1.2

I get the message: The Outbound Domain Enforcement extension has prevented the following From Email Address option(s) from being used as it uses a different domain than the System-generated Mail Settings From Email Address configured at Administer > Communications > Organization Address and Contact Info: mypersonalmail@gmail.com, info@ourdomainandsomething.nl You can add another one here.

[JoeMurray]

Okay, so we need to figure out where it got that gmail address from. Is that what you have put into the Org Address & Contact Info From Email Address, or what you are trying to put into some other form. What form are you filling in when you the error message?

[magnolia61]

It is the emailadres of the account with which I am logged in. I am trying to send a mail using the send email activity from a contacts record But also from the event registration settings I get the message: The Outbound Domain Enforcement extension has prevented this From Email Address from being used as it uses a different domain.

[JoeMurray]

Hmm. I was able to reproduce on a site of ours where my user email used a different email than the one configured for outbound domain enforcement. However, the From email had the valid options previously configured for the site, and when I selected one that had a valid value, and sent the email to the contact it was successfully sent and received. Perhaps we should tone down the 'error' message so that it is a warning, or suppress it altogether as we silently change the default CiviCRM from address in this situation from the current (admin) user's email to one of the emails configured at Admin > CiviMail > From Email Addresses.

Can you try selecting a valid From address on that form, sending, and seeing if it is sent and received?

[JoeMurray]

Heh Richard, we'd like to put out a version of this extension for CiviCRM 4.7 once we close this issue. Were you able to solve you problem, and how? If not, could one of our developers work with you more closely to diagnose the issue?

[magnolia61]

Hi Joe, please reopen. I am happy to spent time helping out testing this extension on 4.6.x and 4.7.x and giving your developers access to my development sites. Status: with the current code from git I still experience the same problem. Trying to send an email from a contact record the from is totally empty because even the confgured 'from'-adress is blocked.

[JoeMurray]

Could you email me at joe dot murray at jmaconsulting dot biz so I can coordinate investigation from our side with you? BTW, Thanks!

[andrimont]

Hello Joe, I enable your extension, and got the similar symptoms. Exactly as @magnolia61 reported :

Trying to send an email from a contact record the from is totally empty because even the configured 'from'-address is blocked.

Regards. Andrimont

[Edzelopez]

Hi @andrimont and @magnolia61,

The purpose of this extension was to block emails used as FROM addresses across the site (in sender receipt configurations, mailings, etc) which did not have the domain same as where the site is hosted. This is useful for servers with a very strict SPF policy to prevent banning or classifying valid emails (having different domains) as spam.

In your case, @magnolia61, you have configured your server's spf to allow sending emails from a different domain. The extension has no way of knowing that, and thus it removed it from the list.

@andrimont, could you tell us a bit more about your issue? Does the FROM address have a different domain than your site? If so, then that would explain why the available selection is blank.

Closing this issue for now, please reopen if you have any additional questions.

Thanks!

[magnolia61]

@Edzelopez: thanx for the info. I think the documentation should be updated to reflect what you say as it now says:

This extension is designed to preserve the email reputation of your server and its IP by ensuring that all outbound email is sent from an address with the same domain as the System-generated Mail Settings From Email Address configured at Administer > Communications > Organization Address and Contact Info (civicrm/admin/domain?action=update&reset=1).

And it shoud read something like: "is sent from the same domain as the domain that is used to access civicrm" (because it ignores the organization address & contact)

Anyway thanx for the clarification. BTW. We have very strict SPF records to only allow sending with the email configured in Org.Adress&contact :-)

[JoeMurray]

Heh @magnolia61. Thanks for your persistence and suggestion that we change the documentation to alert folks to a problem that crops on some installations. I don't quite follow "is sent from the same domain as the domain that is used to access civicrm" (because it ignores the organization address & contact). What do you mean by access in this context?

Can you explain a bit more, if you know, at what level of your system is the 'from' address mangled/stripped?

[magnolia61]

Hi Joe, Thanx for your patience. I´m just one of those semi-programmer civicrm implementers stuggeling to find my way in the woods of JIRA, Github and 3rd party extension land :-)

Edzelopez said:

The purpose of this extension was to block emails used as FROM addresses across the site (in sender receipt configurations, mailings, etc) which did not have the domain same as where the site is hosted.

The readme says:

ensuring that all outbound email is sent from an address with the same domain as the System-generated Mail Settings From Email Address

Our situation is: Our site is hosted at onvergetelijk dot nl But our From Email Adress is onvergetelijkezomerkampen dot nl (also our smtp settings use this domain btw) Result: Trying to send an email from a contact record the from is totally empty because even the confgured 'from'-adress is blocked.

So I believe either there is still an issue or Edzelopez is right and our issue is expected behaviour. But in that case the readme should reflect the hosted domain is used instead of the From adress ;-)

Does that clarify? :-) Dont want to be an hassle... I hope my feedback helps!

[mlutfy]

Maybe I misunderstood, but I also wanted a way to remove personal staff emails from their contacts records, while allowing "from" addresses that were configured by the admin. In my use-case, we trust that the admin added only domains for which the SPF was correctly set.

For example, we may have CiviCRM using "https://client-a.example.org", but their SPF on @example.org was configured to allow their CiviCRM site to send email on their behalf.

Patch:

diff --git a/ode.php b/ode.php
index d9ee788..4ced8aa 100644
--- a/ode.php
+++ b/ode.php
@@ -227,11 +227,22 @@ function ode_suppressEmails(&$fromEmailAddress, $showNotice) {
   // for testing purpose on local
   //$matches[1] = 'jmaconsulting.biz';

+  // Allow domains configured in 'From' admin settings.
+  // The main objective of this extension is to avoid staff personal emails.
+  $domain_from_addresses_raw = CRM_Core_OptionGroup::values('from_email_address');
+  $domain_from_addresses_emails = [];
+
+  foreach ($domain_from_addresses_raw as $key => $val) {
+    if (preg_match('/<([^>]+)>/', $val, $matches)) {
+      $domain_from_addresses_emails[] = $matches[1];
+    }
+  }
+
   $host = '@' . $matches[1];
   $hostLength = strlen($host);
   foreach ($fromEmailAddress as $keys => $headers) {
     $email = pluckEmailFromHeader(html_entity_decode($headers['text']));
-    if (substr($email, -$hostLength) != $host) {
+    if (! in_array($email, $domain_from_addresses_emails) && substr($email, -$hostLength) != $host) {
       $invalidEmails[] = $email;
       unset($fromEmailAddress[$keys]);
     }

Would you accept a patch if there was an admin option to enable/disable this behaviour? (I'm not saying I'll send a patch tomorrow, but if someone else wants to push it forward..) :)

[JoeMurray]

Thanks, Mathieu. This does go against the general purpose of the extension but in a way that makes sense if there is a setting. We're going to put something in.