search

trick77 / dockerflix

Docker-based SNI proxy for watching U.S. Netflix, Hulu, MTV, Vevo, Crackle, ABC, NBC, PBS...
357 stars 58 forks source link

Netflix on Apple TV #31

Open nrj opened 8 years ago

nrj commented 8 years ago

As you probably know, Netflix has started cracking down on proxies and un-blockers. For me this project and setup has been great and working beautifully, until just last week. Now it seems Netflix is geofencing streams!

I've inspected the network traffic in Chrome and I can see Netflix loading video content from nflxvideo.net which unfortunately will now return a 4XX error if you are outside of the USA. If I add the domain to my dnsmasq configuration, the video streams fine. Of course this means I'm proxying the entire stream, but whatever it works and it means that my VPS is not blacklisted.

The curious thing is that, even if I proxy the entire stream, I still cannot access Netflix on my Apple TV (latest gen). My original thought was that it is probably trying to load from a different host. So, I rebooted my Apple TV, started tailing the DNS logs on my Raspberry Pi:

raspberrypi dnsmasq[2338]: query[A] ichnaea.netflix.com from 192.168.1.2
raspberrypi dnsmasq[2338]: config ichnaea.netflix.com is X.X.X.X
raspberrypi dnsmasq[2338]: query[A] www.netflix.com from 192.168.1.2
raspberrypi dnsmasq[2338]: config www.netflix.com is X.X.X.X
raspberrypi dnsmasq[2338]: query[A] api-global.netflix.com from 192.168.1.2
raspberrypi dnsmasq[2338]: config api-global.netflix.com is X.X.X.X
raspberrypi dnsmasq[2338]: query[A] ios.nccp.netflix.com from 192.168.1.2
raspberrypi dnsmasq[2338]: config ios.nccp.netflix.com is X.X.X.X
raspberrypi dnsmasq[2338]: query[AAAA] ios.nccp.netflix.com from 192.168.1.2
raspberrypi dnsmasq[2338]: config ios.nccp.netflix.com is NODATA-IPv6
raspberrypi dnsmasq[2338]: query[A] ios.nccp.netflix.com from 192.168.1.2
raspberrypi dnsmasq[2338]: config ios.nccp.netflix.com is X.X.X.X
raspberrypi dnsmasq[2338]: query[AAAA] ios.nccp.netflix.com from 192.168.1.2
raspberrypi dnsmasq[2338]: config ios.nccp.netflix.com is NODATA-IPv6
raspberrypi dnsmasq[2338]: query[A] ios.nccp.netflix.com from 192.168.1.2
raspberrypi dnsmasq[2338]: config ios.nccp.netflix.com is X.X.X.X

The X.X.X.X is my proxy in the USA and from what I can see, every relevant request is directed there. However I don't see any request that resembles a stream URL... Yet when I try to watch anything on my Apple TV I get a Cannot Play Video (10008) error. HBO, YouTube etc. all work on my Apple TV, yet Netflix works only in my browser

Any thoughts?

jeromeza commented 8 years ago

Hmmm, interesting.

I'm going to test similar from my Roku 3 as Netflix is no longer working and hasn't been for the last +-2 weeks.

I'll update if adding nflxvideo.net fixes the issue.

trick77 commented 8 years ago

Does it still work in a web browser? Because it still works on my Mac/Safari using the Demo Server.

nrj commented 8 years ago

Yes it works in the browser.

nrj commented 8 years ago

Some more info:

If I tail the logs while watching a Netflix show that is available in my area on my Apple TV, I don't see any DNS requests at all during playback. Unlike the browser which sends constant requests to resolve nflxvideo.net. My only conclusion is that the Apple TV app is using IP based stream URLs and since I've confirmed they are geofenced, there isn't much we can do since the Apple TV doesn't support using proxies.

nusnewob commented 8 years ago

Updated my local dnsmasq config to use server instead of address worked for all my devices

server=/netflix.com/x.x.x.x
server=/netflix.net/x.x.x.x
server=/nflxvideo.net/x.x.x.x

From dnsmasq logs

dnsmasq[17880]: query[A] ios.nccp.netflix.com from 192.168.254.103
dnsmasq[17880]: forwarded ios.nccp.netflix.com to 209.177.145.30
dnsmasq[17880]: query[AAAA] ios.nccp.netflix.com from 192.168.254.103
dnsmasq[17880]: forwarded ios.nccp.netflix.com to 209.177.145.30
dnsmasq[17880]: reply ios.nccp.netflix.com is NODATA-IPv6
dnsmasq[17880]: reply ios.nccp.netflix.com is 104.250.139.106
dnsmasq[17880]: query[A] api-global.netflix.com from 192.168.254.90
dnsmasq[17880]: forwarded api-global.netflix.com to 209.177.145.30
dnsmasq[17880]: reply api-global.netflix.com is 104.250.139.106
dnsmasq[17880]: query[A] ipv6_1.lagg0.c072.sjc002.ix.nflxvideo.net from 192.168.254.103
dnsmasq[17880]: forwarded ipv6_1.lagg0.c072.sjc002.ix.nflxvideo.net to 209.177.145.30
dnsmasq[17880]: query[AAAA] ipv6_1.lagg0.c072.sjc002.ix.nflxvideo.net from 192.168.254.103
dnsmasq[17880]: forwarded ipv6_1.lagg0.c072.sjc002.ix.nflxvideo.net to 209.177.145.30
dnsmasq[17880]: reply ipv6_1.lagg0.c072.sjc002.ix.nflxvideo.net is NODATA-IPv6
dnsmasq[17880]: reply ipv6_1.lagg0.c072.sjc002.ix.nflxvideo.net is 104.250.139.106
dnsmasq[17880]: query[A] ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net from 192.168.254.103
dnsmasq[17880]: forwarded ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net to 209.177.145.30
dnsmasq[17880]: query[AAAA] ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net from 192.168.254.103
dnsmasq[17880]: forwarded ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net to 209.177.145.30
dnsmasq[17880]: query[A] ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net from 192.168.254.103
dnsmasq[17880]: forwarded ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net to 209.177.145.30
dnsmasq[17880]: query[AAAA] ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net from 192.168.254.103
dnsmasq[17880]: forwarded ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net to 209.177.145.30
dnsmasq[17880]: query[A] ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net from 192.168.254.103
dnsmasq[17880]: forwarded ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net to 209.177.145.30
dnsmasq[17880]: query[AAAA] ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net from 192.168.254.103
dnsmasq[17880]: forwarded ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net to 209.177.145.30
dnsmasq[17880]: query[A] ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net from 192.168.254.103
dnsmasq[17880]: forwarded ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net to 209.177.145.30
dnsmasq[17880]: query[AAAA] ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net from 192.168.254.103
dnsmasq[17880]: forwarded ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net to 209.177.145.30
dnsmasq[17880]: reply ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net is NODATA-IPv6
dnsmasq[17880]: reply ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net is 104.250.139.106
dnsmasq[17880]: reply ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net is 104.250.139.106
dnsmasq[17880]: reply ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net is NODATA-IPv6
dnsmasq[17880]: reply ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net is 104.250.139.106
dnsmasq[17880]: reply ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net is NODATA-IPv6
dnsmasq[17880]: reply ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net is NODATA-IPv6
dnsmasq[17880]: reply ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net is 104.250.139.106
dnsmasq[17880]: query[AAAA] ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net from 192.168.254.103
dnsmasq[17880]: cached ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net is NODATA-IPv6
dnsmasq[17880]: query[A] ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net from 192.168.254.103
dnsmasq[17880]: cached ipv6_1.lagg0.c016.sjc002.ix.nflxvideo.net is 104.250.139.106
dnsmasq[17880]: query[AAAA] ios.nccp.netflix.com from 192.168.254.103
dnsmasq[17880]: cached ios.nccp.netflix.com is NODATA-IPv6
dnsmasq[17880]: query[AAAA] ios.nccp.netflix.com from 192.168.254.103
dnsmasq[17880]: cached ios.nccp.netflix.com is NODATA-IPv6
dnsmasq[17880]: query[AAAA] ios.nccp.netflix.com from 192.168.254.103
dnsmasq[17880]: cached ios.nccp.netflix.com is NODATA-IPv6
dnsmasq[17880]: query[AAAA] ios.nccp.netflix.com from 192.168.254.103
dnsmasq[17880]: cached ios.nccp.netflix.com is NODATA-IPv6
nrj commented 8 years ago

Using server instead of address breaks all devices for me. My logs look just like yours, except it never gets to nflxvideo.net stream, and my web browser hangs on resolving host... My Apple TV also just spins and times out on Netflix login.

nusnewob commented 8 years ago

Ah forgot to say, you will need dnsmasq running on your remote, so it acts like you upstream DNS for your local dnsmasq,

nrj commented 8 years ago

Sorry, could you please explain a bit more what you mean? Do I need to run a second dnsmasq instance on my server in the US? And if so, why?

nusnewob commented 8 years ago

It acts as a upstream DNS for you local dnsmasq, it forwards DNS queries to remote server instead of your ISP/Google or whatever DNS set in your local network.

nrj commented 8 years ago

I still don't understand because my local dnsmasq is not using my ISP/Google for names related to Netflix. It is returning my server in the US as the answer. And as I mentioned above this works on my browser. Why doesn't it work on my Apple TV?

Furthermore the documentation for this project strongly advises against running open resolvers.

nusnewob commented 8 years ago

address resolves the domain to whatever IP you set, server forwards DNS queries to upstream DNS server. Open resolvers are bad because it answers recursive queries, you can always disable it in dnsmasq or lock it down to your IP. Since Netflix started cracking proxy/vpn, I don't think it works in browser. My guess is nflxvideo.net is using IPv6, and they made the IPv6 to IPv4 tunnel only available in US, which is easy to block proxies.

lbdroid commented 8 years ago

I've never actually seen the domain "nflxvideo.net" in my dnsmasq logs. I have, however, seen a lot of "nflximg.net" and "nflximg.com". Mind you, I don't use a browser -- I use android+chromecast exclusively.

So I'd suggest also adding one of; server=/nflximg.net/dnsserverip (along with the next one on the SERVER's instance of dnsmasq) or address=/nflximg.net/proxyserverip

As far as the "address vs server" discussion, you two are accomplishing the same thing in two manners. Using "server" forwards the dns requests to the mentioned dns server address, which means that you'll be running the dns server remotely, which will be returning the address of that server for matching queries. Using "address" returns the same address, but from the locally running dnsmasq. The end result is, or at least SHOULD be, the same, in that for a particular request, whether the dns is running locally or remotely, it will yield the same response.

@nrj; the two clients will be calling on different domain names. You may want to make sure that the one that is NOT working, isn't asking for a domain name that you aren't handling for it.

nrj commented 8 years ago

@nusnewob finally got around to trying this. Here is my local dnsmasq conf:

server=/netflix.com/x.x.x.x
server=/netflix.net/x.x.x.x
server=/nflxvideo.net/x.x.x.x

And my upstream server (which is x.x.x.x in the U.S.):

address=/netflix.com/x.x.x.x
address=/netflix.net/x.x.x.x
address=/nflxvideo.net/x.x.x.x

Unfortunately it's still working not on Apple TV. Proxy is detected. Working fine in web browser same as before.

What version of dnsmasq are you running?

lbdroid commented 8 years ago

Is the apple actually using your local DNS server? Nothing stopping it from going straight to the IP address of a known public DNS server, like 8.8.8.8. You can set up firewall rules to either block bypassing your local DNS, or redirect servers back to yours.

Have you checked your DNS logs to see if the apple might be requesting domain names outside of those patterns?

nrj commented 8 years ago

Here is something very interesting, if I play around and try different episodes of a show that I know is blocked, occasionally it will load the stream.

Here is what my logs look like when it succeeds:

Apr 12 20:31:08 flix dnsmasq[4905]: query[A] ipv4_1.lagg0.c061.sjc002.ix.nflxvideo.net from 85.177.94.41
Apr 12 20:31:08 flix dnsmasq[4905]: config ipv4_1.lagg0.c061.sjc002.ix.nflxvideo.net is x.x.x.x
Apr 12 20:31:23 flix dnsmasq[4905]: query[A] ipv4_1.lagg0.c061.sjc002.ix.nflxvideo.net from 85.177.94.41
Apr 12 20:31:23 flix dnsmasq[4905]: config ipv4_1.lagg0.c061.sjc002.ix.nflxvideo.net is x.x.x.x

And here is what my logs look like when it fails:

12 20:36:23 flix dnsmasq[4905]: query[A] ios.nccp.netflix.com from 85.177.94.41
Apr 12 20:36:23 flix dnsmasq[4905]: config ios.nccp.netflix.com is x.x.x.x
Apr 12 20:36:23 flix dnsmasq[4905]: query[AAAA] ios.nccp.netflix.com from 85.177.94.41
Apr 12 20:36:23 flix dnsmasq[4905]: config ios.nccp.netflix.com is NODATA-IPv6

@lbdroid yes, if you read my first message I've already verified that all relevant domain queries are going to my server in the US.

lbdroid commented 8 years ago

Those logs are for different domains. Might be helpful for you to show a bigger section of the log.

lbdroid commented 8 years ago

Two things to add @nrj ; 1) Since you mention trying out different videos, it may be that when you try playing a video that is authorized for your country, it works. 2) Netflix has (and I believe that they were already when you started this thread) began making connections direct-to-ip, bypassing DNS altogether. I noticed it first on Android client, but continued working if I casted it to chromecast. A few days ago, they added this to chromecast.

You're going to need to selectively route all networks listed in AS2906.

acarlo79 commented 7 years ago

I am experiencing exactly the same issue on Iphone and Amazon FireTV, all works fine via web. Did anyone find a solution for this?