AdguardTeam / AdGuardSDNSFilter

AdGuard DNS filter
https://adguard-dns.io/
GNU General Public License v3.0
702 stars 126 forks source link

Please block mqtt-mini.facebook.com track domain #1294

Closed ghost closed 1 year ago

ghost commented 1 year ago

Prerequisites

What DNS server do you use?

AdGuard public DNS

Version

-

What DNS upstream(s) do you use in AdGuard apps or AdGuard Home?

-

What DNS filters do you have enabled?

Adguard public DNS

What browser or app do you use?

Other app

Which device type do you use?

Mobile

What type of problem have you encountered?

Missed analytics or tracker

Where did you encounter the problem?

No response

Add your comment and screenshots

Hello, I found that the MIUI of Xiaomi smartphones make constant requests to the Facebook domain "mqtt-mini.facebook.com" for tracking.

In the last 30 days, after monitoring with NextDNS, I noticed that more than 3000 requests were made for that specific domain.

More than 100 requests are made daily.

I tested it on three smartphones to see if it was from a Meta app (like Facebook, Facebook Lite, Instagram, Instagram Lite, Facebook Messenger, Facebook Messenger Lite and Whatsapp) or if it would even crash some of Meta applications.

Two smartphones were Xiaomi. One of them I reset to just factory apps. The third was from Motorola.

"A" is Xiaomi with all Meta apps installed and logged in.

"B" is Xiaomi only with the factory applications, that is, there is no factory installed Meta application, only from Google and Xiaomi.

"C" is Motorola with all Meta apps installed and logged in.

All three smartphones had different DNS from NextDNS to block the domain "mqtt-mini.facebook.com"

Only smartphones A and B were blocked from these domains, which did not change the use of any of Meta's applications. That is, the Motorola smartphone never connected to this domain, despite having all the above applications installed and logged in.

After searching the internet, I verified that it is indeed a Facebook tracker: https://confection.io/trackers/mqtt-mini-facebook-com/

There are reports about this domain: https://twitter.com/Noztox/status/1558069694023622656

https://twitter.com/thefossguy/status/1476090554010468352

https://www.reddit.com/r/blokada/comments/oi0xmi/comment/h4tgvrm/?utm_source=share&utm_medium=web2x&context=3

In the latest link, the user also has Xiaomi's MIUI and has verified that the tracking is theirs.

There is probably an information sharing agreement between Xiaomi and Meta for this tracking to happen.

Therefore, please block this domain mqtt-mini.facebook.com, as it is tracking and monitoring and does not affect Meta applications and Xiaomi smartphones.

Privacy

Alex-302 commented 1 year ago

This is not [only] a tracking domain. FB app will be broken.

ghost commented 1 year ago

@Alex-302 Could you please specify for me exactly what function of the Facebook application this blocked domain fails? Because I have it blocked for more than 30 days and I use Facebook Android app every day and at never had a crash on the smartphones I use.

ameshkov commented 1 year ago

@jakecharlie this domain is used for Facebook Messenger functionality.

It does indeed use the MQ Telemetry Transport protocol hence the confusion.

However, it uses it not for telemetry, but for passing messages. More information here.

One of the problems we experienced was long latency when sending a message. The method we were using to send was reliable but slow, and there were limitations on how much we could improve it. With just a few weeks until launch, we ended up building a new mechanism that maintains a persistent connection to our servers. To do this without killing battery life, we used a protocol called MQTT that we had experimented with in Beluga. MQTT is specifically designed for applications like sending telemetry data to and from space probes, so it is designed to use bandwidth and batteries sparingly. By maintaining an MQTT connection and routing messages through our chat pipeline, we were able to often achieve phone-to-phone delivery in the hundreds of milliseconds, rather than multiple seconds.

ghost commented 1 year ago

@ameshkov Thank you very much for clarifying. Do you have any idea why two Xiaomi smartphones without any Meta apps installed, including Facebook Messenger, connect frequently with this domain? This is what intrigues me the most. Meta apps are not installed as bloatware by default in Xiaomi smartphones.

And at the same time the Motorola Smartphone does not connect to this domain, even though Facebook Messenger is installed on it and working.

It's as if this domain was actually accessed by Xiaomi android OS from the factory and not due to a Meta application installed via playstore (precisely because I did not install any Meta apks).

ghost commented 1 year ago

One more user thread with Xiaomi without any Meta app installed (only whatsapp in this case) and the OS connecting frequently to this domain: https://www.reddit.com/r/nextdns/comments/mqxa0u/blacklisted_facebook_and_still_has_connections/

ameshkov commented 1 year ago

@jakecharlie this is an interesting question.

The problem is that what you're seeing is a DNS query, not an actual connection, and it's hard to say why it's there without inspecting what's inside (which requires rooting the device).

The domain might be used by the Facebook SDK which is used by tons of different apps. If you had AdGuard for Android or a similar app installed, we could at least figure out what app connects to that domain.

ghost commented 1 year ago

@ameshkov Is there any app on the playstore that can do this monitoring for free without rooting the smartphone? I've been trying to do this with NextDNS, but it monitors requests from the entire smartphone and not from each specific application. So I know there's the request, but I don't know where it is

ameshkov commented 1 year ago

@jakecharlie hmm, to be honest, this is a difficult question. Most of firewall apps will only show you IP addresses, most of content blockers apps just filter DNS traffic so they can't be used to figure out which app is doing that.

I think NetGuard should be capable of doing it properly, but part of its functionality is paid so not sure if the free one will be enough.

ghost commented 1 year ago

Confirmed, it is a telemetry domain that came native to Xiaomi smartphones. As I said, it doesn't directly affect how Facebook Messenger works, because the other smartphone I have Motorola and who has Facebook Messenger does not connect with this domain at any time. It is a Facebook telemetry domain that is already installed internally on smartphones from brands like Xiaomi, oneplus, Realme, Poco, Oppo and other brands sold in countries like China, India, Brazil, etc.

https://m.timesofindia.com/gadgets-news/facebook-has-3-hidden-apps-in-most-android-phones-sold-in-india-but-should-you-worry/amp_articleshow/78742140.cms

https://community.oneplus.com/thread/1271623

ghost commented 1 year ago

It is not a specific application that makes these requests, it is the smartphone itself through the package com.facebook.services

Screenshot_2023-05-14-13-14-40-435_com emanuelef remote_capture Screenshot_2023-05-14-13-15-05-839_com emanuelef remote_capture

ghost commented 1 year ago

I am using PCAPdroid from Fdroid to intercept my connection and find out where this is coming from. The only way to remove it is to root the Xiaomi smartphone. So I recommend that Adguard public DNS add this domain to the tracker list.

ghost commented 1 year ago

I discovered that in fact Xiaomi smartphones come pre-installed with this Facebook package (Bloatware). There is no way to uninstall if you are not root. 100% telemetry

Screenshot_2023-05-14-14-32-49-147_com miui securitycenter

ameshkov commented 1 year ago

Could you please try blocking it and see if anything breaks?

Reopening it for now.

ghost commented 1 year ago

I did several tests with Facebook, Messenger and instagram apps (normal and lite versions). None of them connect to this domain, only the pre-installed Facebook services package. Nothing breaks. I've tested everything: newsfeed, messages, calls, stories, reels, watching videos, marketplace, nothing crashes. I'm using Nextdns to block this domain via DNS since I don't have root and even if disabling the package continues to send data from the smartphone: https://imgur.com/a/1PINdl8

Facebook/Messenger/Instagram apps connects to: b-graph.facebook.com graph.facebook.com www.facebook.com rupload.facebook.com edge-mqtt.facebook.com api.facebook.com b-api.facebook.com lookaside.facebook.com snaptu-z.facebook.com web.facebook.com static.xx.fbcdn.net gateway.facebook.com chat-e2ee-mini.facebook.com And some others... but not connects to mqtt-mini.facebook.com

Some prints here: https://imgur.com/a/qj7gLj8

And here from whatsapp: https://imgur.com/a/FNclsBE https://imgur.com/a/xna9Mg5

For me it's the same principle as the dit.whatsapp.net domain (which adguard dns already blocks) that Facebook claims is for good faith purposes but is nothing more than pure telemetry being sent every +/- 10 minutes. The domain mqtt-mini.facebook.com if I monitor with nextdns for a month, as I've done before, there will be more than 3000 requests after 30 days without any Facebook app installed. As it is already pre-installed, either the user makes root, or uninstalls via adb from the computer, both of which are complex for the lay user, or installs a DNS service that already blocks this telemetry.

ameshkov commented 1 year ago

It still bothers me a bit that Facebook itself states that the domain is not a tracking domain.

@Alex-302 we'll probably need to check what actual payload is sent over the connection (rooted device + fiddler or https filtering) just to make sure that we're not blocking something functional.

ghost commented 1 year ago

What I found on the Facebook website about these pre-installed apps is this link: https://tech.facebook.com/artificial-intelligence/2019/9/preinstall-partnerships/

Obviously it says it's for a noble goal, like keeping Meta apps updated (which playstore already does) and help with notifications (which google firebase already does too and which is used by similar social networking and messaging apps). The problem is that because it is already installed by default and without possibility to uninstalling it, it keeps sending telemetry data to facebook servers from all xiaomi, oneplus and several other brands users, regardless of whether they use Meta apps or not. What really happens: the smartphone manufacturers receive money from Facebook to make these applications native so that it can track everyone.

When you read Meta's article about dit.whatsapp.net they don't admit it's telemetry and tracking, they say it's to help understand if the application is up to date, if the messages arrived at the recipient, etc. Always with a noble reason. And in the whatsapp case it is even less serious, because it comes installed with whatsapp and not native to the smartphone itself. So if the user uninstalls whatsapp, the tracking also goes away:

https://engineering.fb.com/2021/04/16/production-engineering/dit/ https://faq.whatsapp.com/538382354780857

ghost commented 1 year ago

I chatted on telegram with the developer of the PCAPdroid application. He suggested the step by step of this link to decrypt the traffic to view the contents of the data sent by the Facebook service.

https://emanuele-f.github.io/PCAPdroid/tls_decryption

I did all the instructions, but the Facebook service package is giving an error, the message below appears:

"The client may not trust the proxy's certificate for mqtt-mini.facebook.com (OpenSSL Error([('SSL routines', '', 'no suitable signature algorithm')]))"

In the link it says that some applications identify the proxy certificate and block the connection, so only with root access it is possible to decrypt the traffic in this case.

Screenshot_2023-05-14-14-14-35-602_com emanuelef remote_capture

Alex-302 commented 1 year ago

In my large logs I see only one DNS record - edge-mqtt.facebook.com Searched for mqtt. Since this request is very rare, I will assume that it is not related to tracking or analytics.

ghost commented 1 year ago

The problem is figuring out exactly what information is being sent. On a Motorola that I have sending information to this domain is also very rare, however 2 Xiaomi's that I have too are sending information to this domain very frequent, as shown in the printscreen above.