mailcow / mailcow-dockerized

mailcow: dockerized - 🐮 + 🐋 = 💕
https://mailcow.email
GNU General Public License v3.0
8.59k stars 1.15k forks source link

Outlook/Office365/Microsoft365 and junk mails #2851

Open patschi opened 5 years ago

patschi commented 5 years ago

This issue is more to document and make people aware experiencing similar issues. Discussion, experiences or any tips to come to a solution might be helpful for everyone.


Office 365 / Outlook is quite special when it comes to get mails from your own mailserver delivered to said providers. In a negative aspect, unfortunately.

The problem There are many reports from users having issues to get serious legit mails delivered to Microsofts' mailing service correctly, even with state-of-the-art non-blacklisted mailservers using latest techniques like DKIM, ARC and strict SPF and being part of their JMRP and SNDS program. In most cases delivered mails are always moved into Junk/Spam folder for absolutely no reason.

Important to notice: This is not limited to mailcow instances overall and is an ongoing issue since a quite long time.

If you have customers at Office 365 or even worse: Outlook.com you should tell them about this issue and migrate them to another service, as they will not be able to receive legit mail from clean mail servers. Business critical mail may never reach their mailbox. This is not the senders problem, this is a serious problem for the recipient and therefore the Microsoft customer. Microsoft seems not to be able to handle their spam filters and tries to mitigate this problem by blocking whole foreign networks.

Solution Unfortunately there is no solution available yet. Several users (including me) tried to get more information and support from Microsoft, but without any noticable improvement nor helpful reply. Apparently Microsoft has no interests at all that their users and companies - relying on Office 365 - gets legit mails of any relevance delivered.

Even analysing all headers on the Microsofts' end after delivering just gives you cryptic headers without any sort of explanation why their considered mails as spam. There are several docs around explaining a few details, but so far they are all either outdated or useless.

Following GitHub issues at Microsofts' docs repository are still pending since a longer period of time to hopefully get some more information:

  1. https://github.com/MicrosoftDocs/OfficeDocs-o365seccomp/issues/743
  2. https://github.com/MicrosoftDocs/OfficeDocs-o365seccomp/issues/442
  3. https://github.com/MicrosoftDocs/OfficeDocs-o365seccomp/issues/409

What can you do? Basically nothing. This might be a workaround.

However you are greatly welcomed to push mentioned GitHub threads above to make Microsoft more aware about this serious issues on their end. If you have more direct connections, use them.

This is going to be continuously updated...

EricThi commented 3 years ago

On my side, scripts work (warning to don't raise limit mail per seconde/minute on gafam) and i have adapt it for send to same mailbox, on different virtual box (for the fun)

/10 cp /usr/local/smtp/templates/joke_jod /usr/local/smtp/tmp/joke_jod && curl -X GET "https://api.jokes.one/jod" -H "accept: application/json" -H "content-type: application/json" -H "X-JokesOne-Api- Secret: api_key" | jq '.. | .text?' | sed 's/null//g' | sed 's/\r/ /g' | sed 's/\n/ /g' >> /usr/local/smtp/tmp/joke_jod && cat /usr/local/smtp/tmp/joke_jod | msmtp -a joke mail1+${RANDOM:0:9}@gmail.com, spa mdebug2791+${RANDOM:0:9}@outlook.com, mail3+${RANDOM:0:9}@hotmail.com && sleep 2 && rm /usr/local/smtp/tmp/joke_jod /15 cp /usr/local/smtp/templates/joke_blonde /usr/local/smtp/tmp/joke_blonde && curl -X GET "https://api.jokes.one/jod?category=blonde" -H "accept: application/json" -H "content-type: application/json " -H "X-JokesOne-Api-Secret: api_key" | jq '.. | .text?' | sed 's/null//g' | sed 's/\r/ /g' | sed 's/\n/ /g' >> /usr/local/smtp/tmp/joke_blonde && cat /usr/local/smtp/tmp/joke_blonde | msmtp -a joke mail1+ ${RANDOM:0:9}@gmail.com, mail2+${RANDOM:0:9}@outlook.com, mail3+${RANDOM:0:9}@hotmail.com && sleep 2 && rm /usr/local/smtp/tmp/joke_blonde /20 * cp /usr/local/smtp/templates/joke_animal /usr/local/smtp/tmp/joke_animal && curl -X GET "https://api.jokes.one/jod?category=animal" -H "accept: application/json" -H "content-type: application/json " -H "X-JokesOne-Api-Secret: api_key" | jq '.. | .text?' | sed 's/null//g' | sed 's/\r/ /g' | sed 's/\n/ /g' >> /usr/local/smtp/tmp/joke_animal && cat /usr/local/smtp/tmp/joke_animal | msmtp -a joke mail1+ ${RANDOM:0:9}@gmail.com, mail2+${RANDOM:0:9}@outlook.com, mail3+${RANDOM:0:9}@hotmail.com && sleep 2 && rm /usr/local/smtp/tmp/joke_animal

after, i have add a "clean" notification and for that, i use my nextcloud with 3 fake account (with mail 1/mail2/mail3) : 1-15/2 9,16 1-5 sh /usr/local/scripts/reunion-g-r.sh 1-30/2 6,18 0,6 sh /usr/local/scripts/reunion-g-o.sh

16-30/2 9,16 1-5 sh /usr/local/scripts/reunion-h-r.sh 31-59/2 6,18 0,6 sh /usr/local/scripts/reunion-h-o.sh

31-59/2 9,16 1-5 sh /usr/local/scripts/reunion-o-r.sh 1-20/2 7,19 0,6 sh /usr/local/scripts/reunion-o-o.sh

example, create a file via webdav with notification : curl -u login:pwd -T /usr/local/scripts/reunion https://nextcloud_domain/remote.php/dav/files/login/Reunion/$(date '+%d-%b-%Y').md

and on calendar app, i have create many appointment with notification by email only

since mai, i have clean my ip and i have configure dane, mta-sts and all are good now...

After, i have test to send mail between mail2 and mail 3 (@hotmail & @outlook.com) => mail are tagged spam (no bad mail, just a default mail )

@mfld-pub yes, if your send your mail via many ip, your reputation is divided by the number of ip on my side, i have disable ipv6 on mailcow : https://mailcow.github.io/mailcow-dockerized-docs/firststeps-disable_ipv6/ and use only one ipv4 dedicated for mailserver

EricThi commented 3 years ago

I never think, use a bin mail for send many mail for increase mail per day .

Example : https://www.mailhazard.com

after create random mail, send many mail (i test it for check if banned with script on cron every minute...)

another services with same idea (sorry, in french website, with services french and english) : https://www.arobase.org/spam/se-proteger/adresse-jetable.htm

milkmaker commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

unixfox commented 2 years ago

not stale

JustinBack commented 2 years ago

I know this is an ongoing issue with problems on microsoft's side, however contacting Microsoft solved this issue for me:

Hello,

My name is XXXXX and I work with the Outlook.com Deliverability Support Team.

We will be looking into this issue along with the Escalations Team regarding IP: (XXXXX). We understand the urgency of this issue and will provide an update as soon as this is available. Rest assured that this ticket is being tracked and we will get back to you as soon as we have more information to offer.

Thank you for your patience.

Sincerely,

XXXXX
Outlook.com Deliverability Support

and not even an hour later the IP has been whitelisted.

Hello,
My name is XXXXX and I work with the Outlook.com Deliverability Support Team.
We have implemented mitigation for your IP (XXXXX) and this process may take 24 - 48 hours to replicate completely throughout our system.
Sincerely,
XXXXX 
Outlook.com Deliverability Support.

So for everybody considering doing a long term mail server, I suggest submitting a ticket. Super fast and E-Mails dont even land in the junk folder.

Still doesn't mean microsoft shouldnt fix it on their end 🤷🏻

CookieCr2nk commented 2 years ago

@JustinBack The Hotmail Sender Support told me that they not see any issue, why i can sending mails to Outlook.com Customers, so i argumented "why i'm not a spammer" and after three mails, they escalated my issue to the Microsoft Customer Support. The Microsoft Customer Support replied quickly and after 24h my IP was unblocked from Microsoft.

I had the issue, that i received no bounce, but the mails never reached the inbox. I think the Hotmal Sender Support can't debug such issues with the Smartscreen filter, so they have to escalate it to the Microsoft Customer Support. After they escalated it to the Microsoft Customer Support the resolution was very quickly and after 24 hours i can now sendign mails to outlook.

grafik

JustinBack commented 2 years ago

Weird, never been contacted by the hotmail sender support, even though I had issues with hotmail.com. All conversations came directly through the microsoft support grafik

2 E-Mails in total 🤷🏻

Our issue was different than yours then, We received bounces from hotmail so it could be an entirely different issue.

For reference, our issue was the S3150 blocklist:

<*******@hotmail.de>: host
eur.olc.protection.outlook.com[104.47.8.33]
    said: 550 5.7.1 Unfortunately, messages from [**********] weren't sent.
    Please contact your Internet service provider since part of their network
    is on our block list (S3150). You can also refer your provider to
    http://mail.live.com/mail/troubleshooting.aspx#errors.
    [AM5EUR03FT011.eop-EUR03.prod.protection.outlook.com] (in reply to MAIL
    FROM command)

Maybe this helps someone with a similiar error message

lecocotier commented 2 years ago

Hello, I'm facing the same issue with Postfix, and can't get any support from Microsoft. So I tried at least to detect the Smartscreen abuse using delivery status notifications and sieve.

f anyof ( header :contains "Content-Type" "report-type=delivery-status", header :contains "Content-Type" "disposition-notification" ){ if anyof( allof (body :contains "X-MS-Exchange-Organization-SmartScreen-Diagnostics", body :contains ["FinalSCL:1","FinalSCL:2","FinalSCL:3","FinalSCL:4","FinalSCL:5","FinalSCL:6","FinalSCL:7","FinalSCL:8","FinalSCL:9"]), allof (body :contains "X-Forefront-Antispam-Report", not body :contains "SFV:NSPM") ){ if header :matches "Subject" "*" { set "subject" "${1}"; } deleteheader "Subject"; addheader :last "Subject" "[WARNING: Microsoft SmartScreen false positive - USE PHONE] ${subject}"; fileinto "Inbox"; stop; } else { setflag "\Seen"; fileinto "Microsoft Smartscreen bug detector"; stop; } }

Appears to work, but probably not cover all the cases.

Dustinlheld commented 2 years ago

DMARC protects users against forged email messages and allows you to manage communications that do not pass SPF or DKIM. DMARC protects your email accounts against spam, spoofing, and phishing.

Adorfer commented 2 years ago

DMARC protects users against forged email messages

so what?

Franselbaer commented 2 years ago

In the meanthime I've had so much trouble with it that I stopped thinking about it and blocking all smtp traffic on all mailservers I get hands on for as8075. Hopefully some day critical mass is reached and they get forced by their customers to provide a proper working email system. Until then, all my mailbox slots stay closed for M$.

FingerlessGlov3s commented 2 years ago

Having to go through Microsoft Support again due to a change of Public IP due to moving from OVH to their SYS (SoYouStart) line but unfortunately can't bring the IP with me.

Thought I would share my experiences again since others may get the same issues and this may help. Before I started my journey with Microsoft support, I made sure rDNS, SPF, DKIM, DMARC is all setup correctly. I also signed up to their SNDS and JMRP. SNDS reports the IP as blocked. I also tried the delist request at sender.office.com but that didn't help either.

So now we need to use the delist form https://support.microsoft.com/supportrequestform/8ad563e3-288e-2a61-8122-3ba03d6b8d75 So first of all the form didn't work and would error, tried different PC, different internet connection, kept erroring, so I waited 24-48 hours and it started working again, so I filled that in eventually, but very annoying. I've seen others report this issue on Microsoft's forum and they start to blame content filters or VPNs, all of which I wasn't using. I also tried 2 different internet connections, but looks like the only solution is to wait and try again.

First response from them is a "please go away", an auto reply it seems, maybe a canned response. Saying the following "...do not qualify for mitigation." and tell you to check your configuration which I've already checked and doubled checked. Gmail can deliver ok, with no warnings or failures in SPF, DKIM and DMARC when checking the headers Gmail adds. So I reply to them telling them I've done all those things and ask for more help.

They came back to me few hours later and said I need to provide proof of purchase of the IP from my ISP. So I send them PDF copy of the invoice and PDF of the email that says the IP have been delivered. I think this pretty reasonable if they need to be able to tell the difference between legitimate requests and fake ones.

So I get another email back from Microsoft hour or so later. Telling me I now need written email from SYS to say I own that IP address. Which is annoying because the invoice and email of delivery proves I own it. So does the reverse DNS record. I logged a support ticket with SYS, which they said the only proof they will send me is the invoice with the IP address in it. If I was a service provider like them, I'd probably say the same, because it has my name, address, and IP on the invoice, no need to do it again, in a less official way through a support ticket.

I then reply back to Microsoft and attach their response as an .eml and the invoice again. I go on to say that they will not provide the confirmation via support ticket and that the invoice is the proof of ownership, I also point out that I have DNS control of the IP address as I have the PTR record set to my domain.

They get back to me about 3 hours later, saying they are going to escalate it. From past experience this means they are going to put a mitigation in. We will be looking into this issue along with the Escalations Team regarding IP: (51.x.x.x).

Another 6 hours later, I get an email telling me the following

My name is Varsha and I work with the Outlook.com Deliverability Support Team.
We have implemented mitigation for your IP: (51.x.x.x) and this process may take 24 - 48 hours to replicate completely throughout our system.

Result! Little bit of emailing and issues with their online form I've finally got the block lifted. I can now deliver to Microsoft provided email again.

G2G2G2G commented 2 years ago

I have 10 small businesses I run their emails. All of them go to spam the first msg to gmail, after it's moved to inbox it will never go to spam again.

Microsoft is pretty similar, occasionally I need to use their mail tools to whitelist an ip again or msg their postmaster and cry. but then they'll work again. Been like this past 20 or so years.

Clete2 commented 8 months ago

It seems like this issue has largely died down. I still run my random email sender and reader script, more out of caution now than anything really.

I rewrote the original script due to performance problems. My new script lives here and I wrote a short blog post about why I rewrote it here. I'm providing these for reference in case anyone runs across this in the future. (don't worry, no gain in it for me in these links -- no ads)

I hope these scripts can help someone who struggles with reputation. Regardless if it is useful, it was a fun project.

Here's to 2024 and no more Microsoft reputation issues!

FingerlessGlov3s commented 8 months ago

Yeah I've had less issues in recent times, guessing they were getting too many false positives.

Here's to a good 2024 email delivery!

shiz0 commented 8 months ago

I can confirm the issue seems to have improved in 2023 with only three occurrences in April, August and November. We will see how it develops in 24.

MineTech13 commented 7 months ago

I can confirm the issue seems to have improved in 2023 with only three occurrences in April, August and November. We will see how it develops in 24.

Still happening just got it the second time this year already and we are not sending anything weird funny even the first time this year we got automatically delisted after 24hours and for the secondtime im waiting to see if it happens again

FingerlessGlov3s commented 7 months ago

I can confirm the issue seems to have improved in 2023 with only three occurrences in April, August and November. We will see how it develops in 24.

Still happening just got it the second time this year already and we are not sending anything weird funny even the first time this year we got automatically delisted after 24hours and for the secondtime im waiting to see if it happens again

What's the volume of emailing that you are sending to Microsoft, just wondering, if it's few or many.