Emails seem to bounce when sent to Google Groups

[eovnu87435ds]

When uploading logs to https://logs.px4.io , you normally get an email from review@px4.io with the log link, a delete link, and some additional info.

Since I share my logs with multiple people, I was hoping to have emails sent to a Google Groups email, but these emails do not seem to reach the group, indicating that there may be something causing an error on the sender side, maybe with email address validation?

I have checked that the permissions of the Google Group were set correctly. I can send emails to it from other external email addresses, and I can also use several online email validator tools to verify that 3rd parties detect the email as valid and reachable.

This happens both with personal Google Groups and Google Groups for Business products. Google's support agents indicates that the issue is on the sender's end.

Looking at app/tornado_handlers/send_email.py, it looks like email errors get printed to console, and the only validation done on the javascript side (plot_app/static/js/upload.js) is that the email includes '@' and is longer than 5 characters.

Steps to reproduce:

1. Go go groups.google.com, log in with a google account (free personal account is fine)

2. create a group. the group email is what you will send mail to. (mypx4logs@googlegroups.com as an example)

3. Set the permissions for "Who can post" to "anyone on the web," make sure "allow email posting" is checked, and uncheck "allow web posting." Set message moderation to "no moderation."

4. Be sure you have your google account as a user within the group. (you should be by default)

5. send a test email from an outside email account to the group email you just made. It should show up in the google group inbox at groups.google.com, and should be forwarded to your group member's inboxes.

After confirming that the google group is set up correctly to receive emails, upload a log to logs.px4.io and enter the google groups email into the email field.

[bkueng]

Thanks for the detailed report. I can reproduce but there's no error.

With debug enabled:

[...]
reply: b'235 Authentication successful.\r\n'
reply: retcode (235); Msg: b'Authentication successful.'
send: 'mail FROM:<review@px4.io>\r\n'
reply: b'250 Ok\r\n'
reply: retcode (250); Msg: b'Ok'
send: 'rcpt TO:<log-test@googlegroups.com>\r\n'
reply: b'250 Ok\r\n'
reply: retcode (250); Msg: b'Ok'
send: 'data\r\n'
reply: b'354 End data with <CR><LF>.<CR><LF>\r\n'
reply: retcode (354); Msg: b'End data with <CR><LF>.<CR><LF>'
data: (354, b'End data with <CR><LF>.<CR><LF>')
send: b'Content-Type: text/plain; charset="us-ascii"\r\nMIME-Version: 1.0\r\nContent-Transfer-Encoding: 7bit\r\nSubject: Log File uploaded (test4)\r\nFrom: Flight Review <review@px4.io>\r\n\r\nHi there![...].\r\n'
reply: b'250 Ok 0101018dac6e5ec5-c83d79ff-233d-4d6f-bb66-e7eeb0c69c08-000000\r\n'
reply: retcode (250); Msg: b'Ok 0101018dac6e5ec5-c83d79ff-233d-4d6f-bb66-e7eeb0c69c08-000000'
data: (250, b'Ok 0101018dac6e5ec5-c83d79ff-233d-4d6f-bb66-e7eeb0c69c08-000000')
send: 'quit\r\n'
reply: b'221 Bye\r\n'
reply: retcode (221); Msg: b'Bye'
[...]

This looks all correct to me, so I'm a bit puzzled by this.

[eovnu87435ds]

I was looking a little bit further into an email I sent to my personal google workspace email. I'm certainly no email expert, so definitely take everything I say with a grain of salt!

Looking through the email headers from log review as opposed to other emails that successfully reach google groups, I did notice two things lacking in the log review headers.

1. The To: field (technically part of the body and not the header) does not exist. I'm pretty sure this has always been optional, since Delivered-To in the header is what really matters. But maybe the lack of a to field is lowering google's confidence in the email being legitimate?

2. I couldn't find a DMARC record for px4.io. Again, I am pretty sure this is optional. (The purpose is to tell the receiving server what to do if SPF or DKIM fail). I don't know if this fixes the issue, but Google does state that your emails rank higher in legitimacy when DMARC is enabled.

[bkueng]

I think you're right, and it's the missing DMARC entry. DKIM is enabled, but the MAIL FROM is 0101018dac62edf7-971d46c4-5728-48d4-83b7-a1b69107096f-000000@us-west-2.amazonses.com, which needs to match the px4.io domain of the DKIM signature in order for DMARC to work. I'll look into getting it set up.

[bkueng]

I enabled it and it is working now with my test group. Can you give it a try yourself?

[eovnu87435ds]

Can confirm, everything working as expected! Thanks for the quick action!!