Iptables redirect doesn't work for ipv6

[gshumihin]

If ipv6 used, adguard unable to filter some https sites (like google.com or instagram.com). Filtering mode should be Local PROXY (auto).

[ameshkov]

We should use ip6tables for this.

[ameshkov]

http://stackoverflow.com/questions/625166/transparent-proxy-for-ipv6-traffic-under-linux

[ameshkov]

http://wiki.squid-cache.org/Features/Tproxy4#IPv6_Support

[ameshkov]

@Revertron don't forget to check if apps detection works ok

[ameshkov]

One more problem with it. Setting IP_TRANSPARENT sockopt requires root access which we cannot acquire in our own app.

The workaround is to use an intermediate TCP proxy which will be started with root access. The only purpose of this proxy is to set IP_TRANSPARENT, set IPv6 redirect rules and then bypass all traffic to the real filtering proxy.

[ameshkov]

It appears to be much more complicated task than we thought in the first place.

Problems:

1. Root access is required to set IP_TRANSPARENT so we should create an intermediate tcp proxy.
2. This intermediate proxy messes with the standard remote endpoint detection algorithm, so we can't filter https this way.
3. IP_TRANSPARENT is present starting from Android L source code, not sure if it even works properly.

Let me reassign it to v2.7 for now.

[Revertron]

For now I've added IPv6 blocking. Browsers and other clients will fallback to IPv4.

[ameshkov]

@Revertron let's move it to a separate issue assigned to v2.6.

Also we need to introduce a new low-level setting so that users could turn this off.

[ameshkov]

Adding @sfionov here, he'll assist with TProxy implementation once we're ready to implement this.

[sfionov]

List of commands for local redirect:

ip -6 route add local default dev lo table 0xad
ip -6 rule add from all fwmark 0xad table 0xad
ip6tables -t mangle -A OUTPUT -p tcp -m owner --uid-owner $UID -j RETURN
ip6tables -t mangle -A OUTPUT -p tcp -j MARK --set-mark 0xad
ip6tables -t mangle -A PREROUTING -p tcp --syn -i lo -s ::1 -j RETURN
ip6tables -t mangle -A PREROUTING -p tcp --syn -i lo -j TPROXY --on-port 12345

[ameshkov]

Isn't it just a list of commands for setting up a transparent proxy?

[sfionov]

Yeah, it is, but it differs from manuals on the internet because we need to redirect locally-generated traffic, not forwarded traffic. It's just for remember how to do it properly.

[sfionov]

Testing notes: It is convenient to test IPv6 on IPv6-only sites like http://ipv6.google.com and http://www.v6.facebook.com, but if filtering doesn't work, it may be noticeable on dual-stack (both IPv4 and IPv6) sites including: http://yandex.ru, http://vk.com, http://www.facebook.com, http://youtube.com. If something goes wrong, connection may not be filtered or may hang with Adguard enabled. In that case, in addition to the log, it will be better to get output of ip -6 rule and ip6tables-save commands from adb shell.

[sfionov]

Log from device where IPv6 failed. Commands shouldn't run and the error reported should be another.

08-04 00:24:20.181 14148-11427/? I/com.adguard.android.f.c: [pool-2-thread-218] executeShellCommands:
                                                            chmod 0755 null
                                                            null /data/user/0/com.adguard.android/cache/socket_receiver lo
08-04 00:24:20.557 14148-11427/? I/com.adguard.android.f.c: [pool-2-thread-218] executeShellCommands result:
                                                            chmod: null: No such file or directory
                                                            sush: <stdin>[2]: null: not found

Can't get transparent server socket, IPv6 redirect will not workjava.io.IOException: Error receiving message on socket: Try again

                                                                at com.adguard.android.filtering.commons.NativeUtils.receiveTransparentServerSocket0(Native Method) ~[na:0.0]
                                                                at com.adguard.android.filtering.commons.NativeUtils.a(Unknown Source) ~[na:0.0]
                                                                at com.adguard.android.f.c.d(Unknown Source) ~[na:0.0]
                                                                at com.adguard.android.e.a.<init>(Unknown Source) ~[na:0.0]
                                                                at com.adguard.android.f.a.a(Unknown Source) ~[na:0.0]
                                                                at com.adguard.android.f.a$1.a(Unknown Source) ~[na:0.0]
                                                                at com.adguard.commons.concurrent.c.run(Unknown Source) ~[na:0.0]
                                                                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133) ~[na:0.0]
                                                                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607) ~[na:0.0]
                                                                at java.lang.Thread.run(Thread.java:761) ~[na:0.0]

[ameshkov]

21:03:14.955 [pool-2-thread-2] WARN  com.adguard.android.f.c - Incompatible iptables executable. Version command output: iptables v1.6.1

[ameshkov]

21:03:15.436 [pool-2-thread-2] ERROR com.adguard.android.f.c - Can't get transparent server socket, IPv6 redirect will not work: 
java.io.IOException: Control message length is too short
    at com.adguard.android.filtering.commons.NativeUtils.receiveTransparentServerSocket0(Native Method) ~[na:0.0]
    at com.adguard.android.filtering.commons.NativeUtils.a(Unknown Source:6) ~[na:0.0]
    at com.adguard.android.f.c.d(Unknown Source:144) ~[na:0.0]
    at com.adguard.android.e.a.<init>(Unknown Source:59) ~[na:0.0]
    at com.adguard.android.f.a.a(Unknown Source:52) ~[na:0.0]
    at com.adguard.android.f.a$1.a(Unknown Source:2) ~[na:0.0]
    at com.adguard.commons.concurrent.c.run(Unknown Source:2) ~[na:0.0]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162) ~[na:0.0]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636) ~[na:0.0]
    at java.lang.Thread.run(Thread.java:764) ~[na:0.0]

[ameshkov]

Merged, closing

[ameshkov]

@sfionov still does not work:

16:28:59.996 [pool-2-thread-2] INFO  com.adguard.android.g.c - executeShellCommands result:

16:29:00.004 [pool-2-thread-2] ERROR com.adguard.android.g.c - Can't get transparent server socket, IPv6 redirect will not work: 
java.io.IOException: Control message length is too short
    at com.adguard.android.filtering.commons.NativeUtils.receiveTransparentServerSocket0(Native Method) ~[na:0.0]
    at com.adguard.android.filtering.commons.NativeUtils.a(Unknown Source:6) ~[na:0.0]
    at com.adguard.android.g.c.d(Unknown Source:150) ~[na:0.0]
    at com.adguard.android.f.a.<init>(Unknown Source:48) ~[na:0.0]
    at com.adguard.android.g.a.a(Unknown Source:52) ~[na:0.0]
    at com.adguard.android.g.a$1.a(Unknown Source:2) ~[na:0.0]
    at com.adguard.commons.concurrent.c.run(Unknown Source:2) ~[na:0.0]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162) ~[na:0.0]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636) ~[na:0.0]
    at java.lang.Thread.run(Thread.java:764) ~[na:0.0]

[ameshkov]

@yochananmarqos need your help on this.

We can't reproduce this issue on our test device. There should be audit lines in the logcat, could you please take a look?

[yochananmarqos]

I browsed a few pages on APKMirror.com while recording a logcat. I saved AdGuard logs right afterword.

adguard_logs_2409_1943.zip 2017-09-24-19-40-31.txt

[ameshkov]

@yochananmarqos it's too short, 2 minutes recorded, nothing about AG startup :(

The only thing we need to be recorded is AdGuard's protection startup. Something goes wrong and AG cannot change the IPv6 listening socket properties.

[yochananmarqos]

I enabled AdGuard protection while connected to T-Mobile IPv6:

2017-09-25-12-30-15.txt

[ameshkov]

@yochananmarqos could you please also grab AdGuard's log?

[ameshkov]

I don't understand how's that possible.

This time I see IPv6 redirection isn't supported on this device record in the log, AG does not even try to set it up.

According to the previous logs, IPv6 redirection is supported and AG did try (unsuccessfully) to set it up.

[yochananmarqos]

adguard_logs_2509_1435.zip

[ameshkov]

@yochananmarqos I got it, the second try after the first unsuccessful does not work.

Here are the steps for recording the logcat log:

1. Exit AG via menu
2. Start it up
3. Grab the logcat log (it should record what we need after the very first startup)

[yochananmarqos]

2017-09-25-15-05-31.txt

[ameshkov]

Great, thank you!

[ameshkov]

@sfionov here's the audit record you were looking for:

09-25 15:05:50.104 W/pool-2-thread-2( 6925): type=1400 audit(0.0:4612): avc: denied { use } for path="socket:[2242340]" dev="sockfs" ino=2242340 scontext=u:r:untrusted_app_25:s0:c512,c768 tcontext=u:r:su:s0 tclass=fd permissive=0

[ameshkov]

@yochananmarqos what type of root do you have?

[yochananmarqos]

I use Magisk, currently v14 stable.

[ameshkov]

Got it, we'll try to reproduce it with Magisk tomorrow.

Not sure if we're able to fix it by the time of the v2.10 release. We should mention it as a known issue in the release notes.

[ameshkov]

Resolved by @sfionov in adguard-android/pull-requests/115

[carlylemiii]

Did this make it in 2.10.104?

[ameshkov]

Yes, it did after all

[carlylemiii]

Then I have missed ads to report, apologies if it isn't explicitly IPv6 that's the issue and this needs to go elsewhere:

androidpolice.com

https://imgur.com/a/wOqOF

https://www.dropbox.com/s/lhch1ovztsuvge2/adguard_logs_2.10.104_2609_1645.zip?dl=0

[carlylemiii]

Also oddly enough I can't access this exact page when enabled

https://imgur.com/ev2QOI5

[ameshkov]

Also oddly enough I can't access this exact page when enabled

Do you have AG certificate moved to the system store? You need to clear Chrome's data to reset the pinning then (pinning is ignored in case of user certs)

[ameshkov]

androidpolice.com

I need to take a look at the log to verify that IPv6 redirect worked in your case.

[carlylemiii]

I do, but it's been there for weeks. Is there a reason it would stop working?

[ameshkov]

I do, but it's been there for weeks. Is there a reason it would stop working?

If there is a single HTTPS request to github.com slipped through AG, Chrome will get the website's pins, remember them and then use for validating the website's cert

[carlylemiii]

If there is a single HTTPS request to github.com slipped through AG, Chrome will get the website's pins, remember them and then use for validating the website's cert

Got it, thanks!

I need to take a look at the log to verify that IPv6 redirect worked in your case.

I think it's definitely an IPv6 issue because I don't get them on Wi-Fi, as I don't have an IPv6 address through my ISP. These ads are only on mobile. Let me know if there are different logs you need!

Btw, for the record, I am currently running SuperSU, not MagiskSU.

Edit: Here's a logcat, but not from the same time frame. Just started AdGuard protection a few minutes ago.

https://www.dropbox.com/s/dzz539gdvwobwy0/2017-09-27-00-38-58.txt?dl=0

[ameshkov]

After all, they all (supersu and magisk) behave differently, reopening it back

[ameshkov]

@carlylemiii @yochananmarqos

Guys, please check the new version: https://github.com/AdguardTeam/AdguardForAndroid/releases/download/2.10.106-rc/adguard_2.10.106_rc.apk

Does it work okay now?

[carlylemiii]

I'm not seeing any ads on Android Police or APKMirror!

[ameshkov]

Great:)

Let's hope we didn't break it for @yochananmarqos :)

[yochananmarqos]

Nope, it's working!

[ameshkov]

Thank you for testing!

It seems we're almost ready to the new AdGuard release guys:)

[androidacy-user]

Hello from six years in the future. Ipv6 filtering is once again broken with root mode, as in it's not filtered whatsoever (with VPN) or times out (on mobile data)

[Versty]

@androidacy-user Hi! This issue is a little bit old. Please create a separate bug report.