SparkDevNetwork / Rock

An open source CMS, Relationship Management System (RMS) and Church Management System (ChMS) all rolled into one.
http://www.rockrms.com
579 stars 352 forks source link

One-Click Unsubscribe not working as intended (v16.3 beta) #5770

Closed kenmulford closed 8 months ago

kenmulford commented 8 months ago

Description

We've configured our upgrade environment (16.3) to use the new one-click unsubscribe feature that will maintain compliance with Google and Yahoo. In our testing we've come across two issues.

  1. In the unsubscribe header of the email, the mailto link is placed before the unsubscribe URL, which we believe is the reverse order of what's needed (comparing this to mailchimp's implementation of this feature as well as the Rock documentation here where it states "Because Rock always includes an API for processing unsubscribe requests (with Enable One-Click Unsubscribe enabled) the email option will probably never be used. ").

  2. We experience an error when attempting to unsubscribe via web browsers because the API endpoint /api/People/OneClickUnsubscribe/ requires POST but the browser is attempting to use the GET method instead.

I'm including these two issues in one report because I believe there's a good chance they could be related. Specifically, we expect that once the order of the items in the first issue are reversed, we may no longer experience the issue in the second item. As it stands, we cannot test browser/mail client functionality of the button because it's only triggering an email.

Mailchimp's implementation:

List-Unsubscribe: <https://example.com/unsubscribe?u=xxx&id=xxx&t=h&e=xxx&c=xxx>, <mailto:notrealdata@unsubscribe.mailchimpapp.net?subject=unsubscribe>
List-Unsubscribe-Post: List-Unsubscribe=One-Click

Actual Behavior

When an email is sent with the one-click unsubscribe feature configured, the user is presented with the "unsubscribe link" but it sends an email instead of prioritizing the URL.

If the URL is directly accessed via the header (like we do with testing in Mailtrap), it won't work because pasting the URL into a browser is GET vs a POST.

When using the API tester, we see a successful confirmation that someone is unsubscribed from a communication list if they were sent an email through a communication list.

Expected Behavior

We expected Apple Mail's "unsubscribe" link to use the URL to unsubscribe us per the Rock documentation and other implementations (e.g. Mailchimp). We were unable to test the URL's functionality since the email address has priority.

Steps to Reproduce

In neither case does the API URL get called.

Issue Confirmation

Rock Version

16.3 Beta

Client Culture Setting

en-US

waynehelsby commented 8 months ago

thanks!!!

kenmulford commented 8 months ago

Fantastic, thank you!

kenmulford commented 8 months ago

@joshuahenninger Heads up - we're testing the 16.3 release now for this feature. The URL does list before the email now (correct). However, we're seeing various behaviors that aren't consistent.

  1. Apple Mail (iOS and desktop OS versions) both show the unsubscribe button
  2. GMAIL - doesn't show the button in the list of emails or on the page where you view the email contents.
  3. Outlook (web and desktop, windows) doesn't show the button/unsubscribe link at all either.

Additionally, we're reviewing headers and there are times that List-Unsubscribe-Post is listed before List-Unsubscribe - and others where the order is reversed. We can't determine any cause for this.

Our core issue is that we don't see the button in gmail when we do see it in Apple, so that makes us concerned that this fix, while helpful, didn't fully implement the feature as needed. What are your thoughts?

nairdo commented 8 months ago

@kenmulford You might also consider this... One might expect to 'always' see the new Unsubscribe button feature show up in Gmail, but it is definitely sporadic/inconsistent. Mailgun confirms this here:

image

waynehelsby commented 8 months ago

we might also be coming up against the issue of our test environment not being used by enough people in gmail to show the button ...

`Gmail doesn't show the Unsubscribe button until it has assessed your domain and decided that you are reliable. This is to stop spammers from inserting the headers and using responses to confirm email addresses.

You will need to also include links within the email body and once a number of people have used those links and you have unsubscribed them, the button will start to show.`

nairdo commented 6 months ago

Since this topic recently came up in a Rock Chat channel, I thought I would repost this here for posterity.


The email address you put into the new field (Request Unsubscribe Email) on the Communication Medium is solely used in the List-Unsubscribe: header along with an unsubscribe URI like this:

List-Unsubscribe: <mailto:info@yourchurch.org>, <https://www.yourserver.org/api/People/OneClickUnsubscribe/EAA....4zK?communicationIdKey=2Km....BR>

(FYI: Your domain must also have a valid/working DomainKeys Identified Mail (DKIM) signature too for any of the one-click unsubscribe feature to work.)

It's up to the inbox provider (Gmail, Yahoo! Mail, etc.) to then handle this. In some situations (not always), the email client might display a special "Unsubscribe" link to the viewer. (See example from Gmail below; again, it might not show the viewer anything special.) And, it might use the URI link to perform the 'unsubscribe'. In other cases, an inbox provider might just send an email to the email address in the List-Unsubscribe header if the viewer performs an unsubscribe action.

Again, all of that handling-behavior occurs on the email client (inbox provider) and they all have different behaviors.

Is the one-click unsubscribe supposed to send us this email to manually unsubscribe or is it supposed to do it automatically?

So, the answer to that question is: it can do either. It's entirely up to the inbox provider (aka "mail receiver" below) to decide. The internet specification (RFC8058) only says:

A mail receiver can do a one-click unsubscription by performing an HTTPS POST to the HTTPS URI in the List-Unsubscribe header...

Also worth noting:

The mail receiver MUST NOT perform a POST on the HTTPS URI without user consent. When and how the user consent is obtained is not part of this specification.

[Update May 7: Above when I mentioned:

"In other cases, an inbox provider might just send an email to the email address in the List-Unsubscribe header if the viewer performs an unsubscribe action. "

-- that means your admin (who handles that inbox) will need to take action, such as unsubscribing the person and/or deactivating the person's email address.]

karenn-ccbc commented 5 months ago

Just adding more details to the conversation (from a thread in #Communications on Rocketchat https://chat.rockrms.com/channel/communications?msg=j72rKiuuSdLyQXWAa ). We just updated from 15.2 to 16.5 and are seeing a few emails come in with no changes to the person's email address on their record and a "null" in the communication id/reference in the email. image

However we also have a staff member who is sending a test email to her staff email address (outlook) and it is getting marked as no mass and then do not email, without her clicking on any unsubscribe link and no emails being sent. No indications in Sendgrid that it is being marked as spam or anything either. image

It seems like we need some kind of indication on their record as to what email they clicked this on or a communication note being added or something.

The email address you put into the new field (Request Unsubscribe Email) on the Communication Medium is solely used in the List-Unsubscribe: header along with an unsubscribe URI like this:

List-Unsubscribe: <mailto:info@yourchurch.org>, <https://www.yourserver.org/api/People/OneClickUnsubscribe/EAA....4zK?communicationIdKey=2Km....BR>

Where do we control this header? In the DKIM settings? or in Rock somewhere? I am thinking that the URL must not be right in ours since the email is coming in with a "null".

nairdo commented 5 months ago

@karenn-ccbc Colleen is going to address the 'staff member' issue you mentioned...

As for the example email from Apple, we do not currently have a way to control the formatting of the emails that the inbox provider (Apple email in this case) uses when they decide to send an email to the address in the List-Unsubscribe: <***> header. But, the mailto: value is controlled globally from the Email medium's "Request Unsubscribe Email" setting:

image

chead4 commented 5 months ago

@karenn-ccbc Hi there - In regards to the staff member who has her email marked as No Mass/Do Not Email without selecting unsubscribe - please send a screenshot of the Person History row at the time her email preference was updated.

This can be found in Person/History and then within the Person History section.

The row should look like this example: image

karenn-ccbc commented 5 months ago

Thanks @nairdo I have had one email from Apple that gave me the name or subject of the communication, all others have said "null" so was wondering if I had missed something in configuration somewhere. Also trying to understand why I am getting the emails only if email is secondary. While the staff workers emails are actually being marked in Rock. Just a learning curve and doesn't feel quite right yet. I have a webinar with sendgrid tomorrow where people from gmail and Yahoo are supposed to talk about the new compliance issues.

karenn-ccbc commented 5 months ago

@chead4 I think I uploaded some to the thread in rocketchat, but will add some tomorrow when I am back at the office.

karenn-ccbc commented 5 months ago

Screenshot_20240529_181835_Chrome

nairdo commented 5 months ago

I have a webinar with sendgrid tomorrow where people from gmail and Yahoo are supposed to talk about the new compliance issues.

@karenn-ccbc You can DM me if you uncover anything new from Sendgrid. We have a few ideas on what might be happening with your staff member situation and we're also trying to figure out what Apple Mail is expecting where they are currently putting the word "(null)" in the emails.

billdeitrick commented 5 months ago

A little late to the party, but we recently had a church with a similar situation where staff members were unsubscribed unintentionally. This was due to their email security appliance opening the links in email headers, and the Email Preference Entry block handling GET requests in the same way as RFC 8058-compliant POST requests. I was about to open an issue but noticed this was fixed on Friday with https://github.com/SparkDevNetwork/Rock/commit/6faaed5a7282e99b9458a5681f7be520ff0faf0c.

Thanks for your work on this @joshuahenninger and core team!