search

AdguardTeam / AdGuardHome

Network-wide ads & trackers blocking DNS server
https://adguard.com/adguard-home.html
GNU General Public License v3.0
23.52k stars 1.73k forks source link

Add an option to suppress Apple Private Relay #3248

Open emax810 opened 3 years ago

emax810 commented 3 years ago

Prerequisites

Issue Details

After install first developer beta of iOS 15, iPadOS 15 and macOS 12 and enable "Privare Relay" from iCloud +, on this devices there are ADS everywhere.

Expected Behavior

Stop ads with enable Private Relay

Actual Behavior

ads everywhere

Screenshots

Additional Information

If I turn-off "Private Relay", AdGuard Home works without problems .

ainar-g commented 3 years ago

Hello. This Apple Private Relay service looks a lot like a VPN. Are you entirely sure that your devices continue using AGH as their only DNS server? Can you please provide the logs of requests? Thanks.

emax810 commented 3 years ago

Hi,

i see these DNS requests for devices with "Private Relay" enable :

nothing else :(

emlimap commented 3 years ago

While there isn't much detail on the inner workings of Private relay. What we know so far

  1. It is limited to Safari and doesn't work as a whole device VPN, at least in its current form.
  2. It uses Cloudflare Wrap, at least for exit node part of the connection - https://twitter.com/wongmjane/status/1402027672767664128
  3. Apple seems to have implemented a lite version of tor. For private relay there are 2 nodes, first node which knows the IP of the user but not the website being accessed. This server then relays the traffic on to second server(most likely cloudflare based on point 2) which knows the website being accessed but not user's IP. Tor requires minimum of 3 nodes.

I don't have access to developer build of iOS 15 to verify this but from what I gather it sounds like it tunnels/proxies Safari traffic through their servers and eventually to Cloudflare. Safari is most likely doing remote DNS bypassing the DoH settings on the device. I presume Safari content blockers would still work as they are applied by Safari itself.

These 2 domains look new as I don't see them on a network with devices running both iOS 14.6 & iPadOS 14.6. Second domain is a CNAME to first one.

mask.icloud.com
mask.apple-dns.net
ainar-g commented 3 years ago

I have a strong feeling that this isn't really an AGH issue, but we'll still need to investigate it and probably add it to the documentation. Hopefully, Safari has some kind of internal setting for this kind of thing.

agneevX commented 3 years ago

DNS requests are made via the Private Relay so I don't think any software on the device can interfere without breaking the functionality (if that's even possible).

Apple says that Private Relay will also include DNS queries

From The Verge: https://www.theverge.com/2021/6/10/22526881/apple-icloud-plus-privacy-subscription-services-revenue-wwdc-2021

emax810 commented 3 years ago

more documentation from Apple Developer :

Transcript

iCloud Private Relay is a new service that prevents networks and servers from monitoring user activity across the internet. And it's available as part of every iCloud+ subscription. Private Relay not only protects users when they're browsing the web, but also adds protection to the traffic generated by your app to make sure you're not unintentionally leaking user information or exposing users to security attacks. Today, you'll learn what iCloud Private Relay is and how it affects your app; how to ensure that your app works great with Private Relay; how to prepare your websites and servers; and, lastly, how to manage a network and monitor your traffic when Private Relay is enabled. Private Relay is built into iOS and macOS, so you don't need to do anything to adopt it from your app. It's also important to understand that it won't always be affecting your app. It will only apply when a user is an iCloud+ subscriber and has Private Relay enabled. You're probably wondering, What does it actually do? Here's how things currently work without Private Relay. When someone accesses the internet, anyone on their local network can see the names of all of the websites they access based on inspecting DNS queries. This information can be used to fingerprint a user and build a history of their activity over time. No one should be able to silently collect all of this information, whether it's a public Wi-Fi operator, another user on the network, or an internet service provider. When connections reach the servers that run websites, those servers can see the user's IP address. This allows the servers to determine user location without explicit permission. Even worse, the servers are able to fingerprint user identity and recognize users across different websites, even when tools like Intelligent Tracking Prevention in Safari are preventing correlation via cookies. These are big problems for user privacy, and in order to fix them, we need a new approach that has privacy built in by design. **iCloud Private Relay adds multiple secure proxies to help route user traffic and keep it private.** The proxies are run by separate entities. One is Apple, and one is a content provider. Now, when someone accesses the internet, only the client IP address is visible to both the network provider and to the first proxy. The second proxy only sees the name the user is requesting and uses that to build the connection to the server. It is critical to note that no one in this chain -- not even Apple -- can see both the client IP address and what the user is accessing. The opportunities for fingerprinting have been removed. This is privacy by design. Private Relay uses the latest transport protocols and privacy-preserving authentication to ensure that every transaction is both secure and fast. You can learn more about this technology in the “Apple's privacy pillars in focus” session. Private Relay is focused on securing the most sensitive traffic on the system without impacting user experience. In iOS 15 and macOS 12, Private Relay will apply to all web browsing in Safari, all DNS name resolution queries, and a small subset of traffic from apps. Specifically, this will include all insecure HTTP traffic, such as TCP port 80. If your app provides a content filter or a parental controls filter, it will still see traffic before it goes through Private Relay, so you can apply your filters just as before. You can learn more about this in the "Meet the Screen Time API" session. Not all networking done by your app occurs over the public internet, so there are several categories of traffic that are not affected by Private Relay. Any connections your app makes over the local network or to private domain names will be unaffected. Similarly, if your app provides a network extension to add VPN or app-proxying capabilities, your extension won't use Private Relay and neither will app traffic that uses your extension. Traffic that uses a proxy is also exempt. Next, let's cover what you need to do to make sure your app is ready to work well with Private Relay. The great news is that for almost every app, you don't need to do anything new! Private Relay will just work. However, there are some best practices you should know about. Private Relay works no matter what networking API you're using. For several years, we've recommended that your apps use modern APIs such as URLSession and NWConnection. Now, you have one more reason to adopt them throughout your apps. These APIs provide you the best tools to understand how Private Relay is applying to your traffic. If you use URLSession, you can use the Network Xcode Instrument to inspect your tasks, even when they are going through Private Relay. You can learn more about how to do this in the "Analyze HTTP traffic in Instruments" session. And you can use metrics APIs in both URLSession and Network.framework to understand when your connections use Private Relay. In TaskTransactionMetrics, you can check to see if your task used a proxy. In NWConnection's EstablishmentReport, you can also inspect the timings of DNS name resolution and each stage of the proxied connection establishment. Private Relay takes a big step towards making unencrypted and insecure HTTP connections a thing of the past. You can join in this effort too! If you're still using insecure HTTP -- that's connections on TCP port 80 -- now is the time to change. When Private Relay is enabled, these insecure connections will be proxied, which protects them from attackers on a local network between the client and the proxy. However, it’s still best to have your connections be secure end to end for all users. To do this, make sure your server supports TLS and change your URLs from http:// to https://. Your app may have exceptions for App Transport Security to allow insecure traffic. You can look at this list to audit what insecure traffic your app uses and remove these exceptions. If your app provides functionality based on location, this is also a great time to make sure you're using APIs from Core Location instead of relying on servers inferring location from IP addresses. IP address geolocation is often unreliable and inaccurate. Core Location allows you to specify the precision of the location you need and is based on explicit user permission. You can learn more about the latest enhancements to location access in the “Apple's privacy pillars in focus” session. Once you've made sure your app is ready for Private Relay, try it out! Sign into an iPhone, iPad, or Mac with an iCloud+ subscription, and make sure Private Relay is enabled in the iCloud section of the Settings app or in System Preferences. You can provide feedback with the tag "iCloud Private Relay" in the feedback app. Now I'll hand it off to my colleague, Delziel. Delziel Fernandes: Thanks, Tommy. Hi, I'm Delziel Fernandes, and I'll be talking about how to prepare your server and manage your network with Private Relay. If you have servers that your app relies on, or that user access in Safari, there are a few things you can do to prepare for Private Relay. Your servers can identify connections that come in using Private Relay by recognizing the proxy IP addresses. These proxy IP addresses may be shared by many users within a region. Each address is mapped to a specific city or region. So if you apply the correct geo IP mapping databases, your servers will still have the relevant information. Private Relay guarantees that users can't use the system to pretend to be from a different region, so you can continue to enforce region-based access restrictions. Details about the proxy IP addresses will be available as an article associated with this session. Now let's take a look at network connections from devices using Private Relay. When a device tries to access a server, it first sets up a network connection to the ingress proxy. This connection is set up using an IP address assigned by the network provider. The device then uses the ingress proxy to forward network requests to the egress proxy using the ingress proxy IP address. The egress proxy then forwards these requests to the destination servers by choosing an IP address that maps to the device's city or region. In general, your servers and websites should stop solely relying on client IP address to determine user location or identity. If you need location access, consider requesting the user's location explicitly, and only at the granularity you need. If you need to identify users, request a login or some other form of explicit identification rather than assuming that the IP address is tied to the identity. Last, let's cover some tips for managing your network when Private Relay is in use. If you're running a packet trace on your local network when Private Relay is in use, you'll see some new traffic patterns. You'll now see a lot more traffic running on UDP port 443. This is QUIC -- or HTTP/3 -- traffic that's being used to communicate with the Private Relay proxy. You can make sure your traffic works well by allowing UDP port 443 on your network and by making sure your routers or Network Address Translators are tuned to handle it well. You'll also see fewer cleartext UDP DNS queries on your network. Let's go over a typical network connection request from a device when Private Relay is not in use. When a device tries to access a server, it first sends a DNS query for the hostname of the server. Once the hostname is resolved to an IP address, the device then connects to the IP address of the server using a transport protocol like TCP. The device then performs a TCP three-way handshake with the server followed by a TLS exchange to set up a secure connection with the server. But as you can see, without Private Relay, the hostname of the server and the IP address of the device connecting to the server are visible by simply observing packets on the network. With Private Relay, the device first sets up a connection to the ingress Proxy using QUIC, or HTTP/3. In packet captures, you will notice UDP packets sent to port 443 of the ingress proxy. Once the network connection is established to the ingress proxy, access to a server is secured within the connection to the ingress proxy. On the server end, there is no change in protocol. The TCP/TLS exchange is similar to traffic without Private Relay. The only difference is that the server sees the incoming connection from a proxy IP address instead of a device's IP address. Most networks don't need to audit or monitor all user traffic. However, if you run an enterprise or school network, your network may have policies that require intercepting all traffic. To support this use case, you can block the hostname of the iCloud Private Relay proxy server. Then, when a device connects to your network, the user will receive a prompt indicating that Private Relay is blocked on the current network. They can then choose to either disable Private Relay for that network or switch networks. Information about the proxy hostname will be available as an article associated with this session. For parental controls, the best solution is to use content filter APIs provided by NetworkExtension framework. This allows traffic to be audited on device even when Private Relay is enabled. For more information on content Filter APIs, watch "Meet the Screen Time API" and "Network extension for the modern Mac" sessions. [Meet the Screen Time API](https://developer.apple.com/videos/play/wwdc2021/10123/) [Network Extensions for the Modern Mac](https://developer.apple.com/videos/play/wwdc2019/714/) To wrap things up, use modern secure networking APIs like URLSession in your apps; test your apps with Private Relay; and make sure your servers and websites work well when they receive connections routed through Private Relay.
agneevX commented 3 years ago

Damn. GitHub really needs to add a way to collapse a comment.

ameshkov commented 3 years ago

Well, probably you could configure AGH to be an encrypted DNS server (DOH or DOT) and route DNS traffic there by installing a DNS profile.

Final-Hawk commented 3 years ago

You can just disable it just for the network. Go to your wifi settings and press it 'i'. Scroll down and disable private relay. I don't think adguard home or other solutions will be able to bypass this easily.

agneevX commented 3 years ago

Well what’s the point of paying for the feature if you’re going to end up disabling it?

Final-Hawk commented 3 years ago

Now this is just me, but I already pay for iCloud storage, so i'm not paying extra for iCloud relay. However, because you can disable it on specific wifi networks, you can still get the benefits when on public wifi etc.

BirdInFire commented 2 years ago

Now this is just me, but I already pay for iCloud storage, so i'm not paying extra for iCloud relay. However, because you can disable it on specific wifi networks, you can still get the benefits when on public wifi etc.

true but generate a problem, in my case I have a DOH profile who enable DOH to AdHome when outside of my network, so your config will bypass this dns filtering when outside, and the app dns request will remain unsecured.

Dubz commented 2 years ago

Apple's private relay includes DNS queries for privacy. They are sent as DoH requests to the specified domains that you are seeing. The only way you could get the best of both worlds is if Apple decides to let users use a custom DNS server rather than force their own. Aside from that, you're pretty much stuck with one or the other. You could block DoH requests to those domains (or DNS lookup on said domains), however that'll just kill off your DNS entirely and you'll be stuck with direct IP requests only (which is not ideal obviously).

agneevX commented 2 years ago

Per /r/PiHole:

https://reddit.com/r/pihole/comments/ofkh2t/_/h4dhorc/

joshqou commented 2 years ago

If you want to use Private Relay, or any other kind of proxy/VPN service, then you should use AdGuard for Mac or AdGuard for iOS. If you're going to use a VPN there really isn't that much a DNS adblocker can do, aside from trying to block it by default and causing a flood of issues complaining about said block like Pi-Hole

imo AdGuard Home should provide a toggle in settings to block Private Relay, just not by default.

ameshkov commented 2 years ago

Can it be blocked on the DNS level?

joshqou commented 2 years ago

@ameshkov Yes. https://developer.apple.com/support/prepare-your-network-for-icloud-private-relay

ameshkov commented 2 years ago

Oh, cool, thank you!

ameshkov commented 2 years ago

@ainar-g we should add a new setting that suppresses Apple Private Relay.

I suggest enabling by default as Private Relay defeats the purpose of AdGuard Home. It should be possible to enable or disable this setting on the per-client basis.

Please assign this feature to a milestone.

danieletorelli commented 2 years ago

@ainar-g we should add a new setting that suppresses Apple Private Relay.

I suggest enabling by default as Private Relay defeats the purpose of AdGuard Home. It should be possible to enable or disable this setting on the per-client basis.

Please assign this feature to a milestone.

I'd suggest adding it in the blocked services.

ameshkov commented 2 years ago

Well, blocked service is more of a "parental control" type of feature, and this one is of a greater scope.

joshqou commented 2 years ago

Please DO NOT block private relay by default, pi-hole did this and it was a pain in the ass to figure out why my iPhone and Mac suddenly weren't connecting to private relay. It should be optional like the rest of the blockable services are

ameshkov commented 2 years ago

Well, the problem with private relay is that it's not just a "service", it basically circumvents AdGuard Home.

I am not sure how Pi-Hole did this, but we'll add a setting that will be visible in the UI, would it be enough to figure out what's disabling it?

Dubz commented 2 years ago

DoH circumvents most all DNS servers, and that's what Apple's private relay is. It's a DoH service. If it can resolve its DoH domain(s) using regular DNS (via AdGuard, for example), then it can look up everything else through the relay via DoH. You would simply need to block those domain names that run the DoH relay, and it will never be able to get to it to perform DoH queries. That's all PiHole would have had to do.

I simply use Cloudflare for teams DNS and filter out DoH domains with it (there's a setting). That way it does Apple, Google, and any other known providers.

BirdInFire commented 2 years ago

Well, the problem with private relay is that it's not just a "service", it basically circumvents AdGuard Home.

I am not sure how Pi-Hole did this, but we'll add a setting that will be visible in the UI, would it be enough to figure out what's disabling it?

me i "vote" to have into service section a section like "integrated vpn" who list all "app vpn" like the vpn of brave / firefox / apple relay/ etc who are embedded in software / os, and where their domain name are known.

and (with a warning like you have said) when on the main page we enable the XXX API to let the parents known they must enable this to "prevent" their child to circonvent it.

ainar-g commented 2 years ago

@ameshkov, I really don't think that that should be enabled by default.

I also agree that the feature sounds like something that should be in the Blocked Services section.

danieletorelli commented 2 years ago

I also agree with who says that it shouldn't be enabled by default and again, IMHO, blocked services sounds the perfect fit for it.

For context, in case it can be useful to someone else, those are my custom rules that I was using so far to block it till when this will be implemented:

# Disable iCloud Private Relay
||mask.icloud.com^$important,dnsrewrite=NXDOMAIN;;
||mask-h2.icloud.com^$important,dnsrewrite=NXDOMAIN;;
kuduacz commented 1 year ago

Strange things. Dosable iCloud private relay, and still same effect. No iPhone route in AGH.

BirdInFire commented 1 year ago

Strange things. Dosable iCloud private relay, and still same effect. No iPhone route in AGH.

for me i've installed tailscale iphone & server, so like that (under dns, override local dns and put the tailscale ip assigned to your server) and you dnsrequest will be forced by the vpn (wireguard) to your server (FOSS) and if you don't enable the exit node feature) only DNS request will be pushed by the vpn.

kuduacz commented 1 year ago

Let me explain my situation. Router (Tenda nova 5) configured with AGH DNS. My devices (i mean smarphones and MacBook) configured with AGH DNS too. And it works well (without my iPhone) With that config i ser how many request generale each device and i will not chang it. It have to be like now.

I disabled iCloud private relay and i was expecting my iPhone back to AGH route. But its not. Its only generate few requests.

But, sites which i visiting i see in AGH request just by router, not my iPhone. When my wife enter to some sites its under her smartphone requests in AGH. How to solve my situation?

I use Tailscale to connect homeassistant when not in local network (out of home).

gustakasn0v commented 5 months ago

I also agree with who says that it shouldn't be enabled by default and again, IMHO, blocked services sounds the perfect fit for it.

For context, in case it can be useful to someone else, those are my custom rules that I was using so far to block it till when this will be implemented:


# Disable iCloud Private Relay

||mask.icloud.com^$important,dnsrewrite=NXDOMAIN;;

||mask-h2.icloud.com^$important,dnsrewrite=NXDOMAIN;;

Thanks for sharing! But as of today, these don't seem to be working anymore. I used https://d3ward.github.io/toolz/adblock.html to verify if AdGuard Home was still filtering, and https://vpnapi.io/relay-detection to know if the iCloud Relay was active on my iPhone; despite the filtering rules, the iCloud Relay stays active somehow.

So far the only reliable way I've found is to go on the WiFi network settings and toggle "Limit IP address tracking" off. This of course isn't ideal, as I have to do this for every Apple device in my network.

danieletorelli commented 5 months ago

Thanks for sharing! But as of today, these don't seem to be working anymore. I used https://d3ward.github.io/toolz/adblock.html to verify if AdGuard Home was still filtering, and https://vpnapi.io/relay-detection to know if the iCloud Relay was active on my iPhone; despite the filtering rules, the iCloud Relay stays active somehow.

So far the only reliable way I've found is to go on the WiFi network settings and toggle "Limit IP address tracking" off. This of course isn't ideal, as I have to do this for every Apple device in my network.

@gustakasn0v:

I toggled that option long time ago as well, but those rules still seem to be valid and should be checked by the device as stated in the Apple Developer portal page.

BirdInFire commented 5 months ago

Me switching the refusal answer to nxdomain did the trick

gustakasn0v commented 5 months ago

I also agree with who says that it shouldn't be enabled by default and again, IMHO, blocked services sounds the perfect fit for it.

For context, in case it can be useful to someone else, those are my custom rules that I was using so far to block it till when this will be implemented:


# Disable iCloud Private Relay

||mask.icloud.com^$important,dnsrewrite=NXDOMAIN;;

||mask-h2.icloud.com^$important,dnsrewrite=NXDOMAIN;;

Thanks for sharing! But as of today, these don't seem to be working anymore. I used https://d3ward.github.io/toolz/adblock.html to verify if AdGuard Home was still filtering, and https://vpnapi.io/relay-detection to know if the iCloud Relay was active on my iPhone; despite the filtering rules, the iCloud Relay stays active somehow.

So far the only reliable way I've found is to go on the WiFi network settings and toggle "Limit IP address tracking" off. This of course isn't ideal, as I have to do this for every Apple device in my network.

Eureka! It didn't work immediately, but a couple hours later I got a prompt on my iPhone telling me my network wasn't compatible with iCloud Private Relay, and prompting me to disable it.

So this means this method works! Just FYI it may take a while for existing devices to detect it.