Cosmetic filtering doesn't work with wireguard running.

[Rtizer-9]

Please answer the following questions for yourself before submitting an issue.

• [X] I am running the latest version
• [X] I checked the documentation and found no answer
• [X] I checked to make sure that this issue has not already been filed

AdGuard version

4.0.601

Environment

NA

Root access

• [X] Yes, I have it.

What filters do you have enabled?

No response

What Stealth Mode options do you have enabled?

No response

Issue Details

Steps to reproduce:

1. Turn on automatic proxy filtering mode (root) in adguard.
2. Turn wireguard on (applies to both kernel mode and userspace mode).
3. Whitelist wireguard from adguard routing as advised for all such apps based on different vpn protocols.
4. Install adguard system certificate using magisk to enable https filtering in all apps.
5. Use apps, specifically browsers and the ones based on system webview.

Expected Behavior

DNS, HTTPs and Cosmetic filtering should work in all apps on both IPv4 and IPv6 network as stated in redesigned nightly release notes with adguard correctly identifying which web request belongs to which app.

Actual Behavior

Cosmetic filtering STOPS working in apps with filtering log showing all entries as DNS entries.

Screenshots

No response

Additional Information

I was about to file the issue earlier then suddenly found out that filtering starts working perfectly when IPv6 filtering is disabled in adguard advanced setting leading to completely disabling IPv6 in whole system. But this leads to network unstability within few minutes.

I also found that routing adguard dns filtering using another dns based app made the setup work as expected. The app has its several options including choosing between its own iptables or system iptables.

All these concludes that adguard root automatic proxy needs to make the iptable reconfiguring part more compatible with wireguard so that all features work just like they're supposed to work in automatic proxy mode as stated in release notes.

[sxgunchenko]

Cannot reproduce the issue. @Rtizer-9 could you please collect debug log. One more thing that could help is an iptables dump: it can be collected with iptables-save/ip6tables-save or, in case it's unavailable, iptables -vL/ip6tables -vL.

[Rtizer-9]

https://wormhole.app/MvyQ1#EtmM6mjAPgOAtDY59rcyIA

(All logs along with both iptables and ip6tables dump)

(Link will be alive for 24 hours)

The issue occurred from 8.29 to ~8.31PM.

---

Let me walk you through the exact environment the phone was in when I captured the log and the issue occurred again :

1. I've started putting ag in VPN mode since root proxy conflicts with wireguard turned on simultaneously and results in deep sleep lost with overflow partial wakelock shown.

2. So for collecting logs I turned off as much as apps I can > went to ag settings then turned on both ipv6 and root proxy > set logging level to required (didn't even open let alone make any single change in the official wireguard Android app > opened google news and ads are back.

3. During all this time wireguard was working in the background. My wireguard profile's name is wgcf. Wireguard is using kernelspace module not userspace go backend (which creates a VPN) during this time.

I've been facing this issue for a long time now. First I found that after disabling ipv6 things work but then even after that whenever connection changes wifi goes down and connectivity changes to phone network then internet stops working and after fiddling with wireguard switch for like ~20 times, I finally get ag and wireguard work together.

The way I see it, there needs to be a dedicated rule in ag iptable modification when it is in root proxy mode to let the wireguard app/module (for both kernelspace and userspace mode) work without any conflict.

All the above logs were captured while the internet was working but cosmetic filtering wasn't. When I turn wireguard switch on and off, it takes a lot of fiddling with all the settings to make them work together again as I mentioned above and during that iptables were also different when I checked them once earlier. It's important to note that while logging was on that "no internet" situation didn't occur even once fortunately.

If you need any other information please comment.

[Rtizer-9]

There is some interesting information given in this link related exactly to this case. It even goes on to mention how turning on IPv6 will cause all dns queries to bypass the filtering app.

[Rtizer-9]

@sxgunchenko Have you received the logs or do I need to upload it again since the link has expired?

[sxgunchenko]

@sxgunchenko Have you received the logs or do I need to upload it again since the link has expired?

oops, sorry, I missed your message and now the files are unavailable

[Rtizer-9]

@sxgunchenko https://wormhole.app/d0BBK#uMMGNF2CCnqCO_PtaGb_Dg

[sxgunchenko]

Great, thank you! I'll take a look.

[Rtizer-9]

Hi @sxgunchenko, is there any update on this?

I can see many other similar issues with both wireguard and ipv6 being filed with none of being answered. I'm still facing that wireguard issue on a daily basis.

[sxgunchenko]

I didn't find any issues in the configuration. Some more time is needed for investigation.

[Rtizer-9]

I've personally seen many other people facing the IPv6 error and wireguard error too.

I think it would be better to replicate the setup on a spare rooted device if you have and see how ag and wireguard plays together to get a first hand experience of how they conflict.

[sxgunchenko]

I have already tried but it did not reproduce. Maybe, my setup did not quite match yours. I'll take another try.

[ameshkov]

I have to close this as "cannot reproduce" as we're out of ideas :( We'll reopen it if there's new relevant info.

[Rtizer-9]

@ameshkov ipv6 and cosmetic filtering is not much of a concern for now since the network isn't working at all when Wireguard is being used with Adguard.

I tried to make them work a lot but nothing helps. It's all a matter of which is turned on first and then writes the required entry in iptables.

I need to do a lot of things, like turn off wifi, mobile, turn them back on, turn Wireguard on/off and turn ag on/off, force update in ag to trigger network.

Once you somehow get them to work any single time connectivity changes like power off or leaving home resulting in no wifi then again same rinse repeat.

Sometimes it works, some it doesn't no matter how long you experiment.

My understanding is that in root mode both the apps are using iptables/ip6tables and since both apps aren't aware about other's presence and handling or network they end up writing rules which makes the one other incompatible resulting in total network loss.

The problem persist with Wireguard VPN mode too though.

I have seen rules being created in termux with iptables command whenever any/both of them are turned on.

Can you please look into it through that route?

[Rtizer-9]

#4652 and #2759 are having the exact same issue.

[ameshkov]

My understanding is that in root mode both the apps are using iptables/ip6tables and since both apps aren't aware about other's presence and handling or network they end up writing rules which makes the one other incompatible resulting in total network loss.

Yes, most likely this is what happens.

Actually, the fact that in #4652 it helps to restart AdGuard immediately after establishing a VPN connection is encouraging. Most likely it would be possible to react to a network change event and re-create the iptables rules in the case when the main interface belongs to Wireguard.

Let's reopen this issue and take another look.

[Rtizer-9]

@ameshkov @sxgunchenko I've some good news, I've kinda figured out the issue that's stopping adguard from redirecting all packets to itself and I'm mostly able to use this setup with almost 90% stability.

I think it needed to be mentioned that I'm trying to use cloudflare-warp specifically here in official wireguard app. To reproduce you can simply -

1. Generate a cloudflare-warp vpn profile using this tool.
2. Use that generated profile and import it into official wireguard app.
3. Grant root permissions to both Ag and Wireguard so that ag can use automatic proxy and wireguad uses kernel backend. (also install CA certificate into system using adguardcert module)
4. Exclude wireguard from ag filtering as it has been normally suggested for vpn apps in your kb.
5. Observe filtering in all the cases -
   • Ag (automatic proxy) - Wireguard Kernel Mode
   • Ag (vpn mode) - Wireguard Kernel Mode
   • Ag (automatic proxy) - Wireguard Userspace Go backend (Vpn).
   • Both of them can't be used in vpn mode simultaneously obviously.

• Now the Wireguard app has some interesting settings out of which 75% directly/indirectly should affect traffic routing if you're using it with Ag, one of them is "Allowed IP" which defines the IP ranges for which a peer will route traffic.

Official Cloudflare warp clients have all local address excluded from their routing so ideally we should also be whitelisting them wireguard official app when we're using warp in it. Official wireguard app also has a dedicated option to exclude private IPs.

So I calculated those addresses using this WireGuard AllowedIPs Calculator with some ISP addresses too to let volte/vowifi work just in case.

Now, when you'll use this setup we'll still see ads bypassing Ag even if ag is turned on. Now I'd linked something above which says ipv6 can bypass dns apps so I thought let's stop ipv6 routing through wireguard and I removed ipv6 addresses I got above which I had put in "Allowed IP". Ag team also mentioned specifically about automatic proxy mode having some issue with ipv6 filtering earlier.

And now ads are properly filtered as far as I can test in both vpn and automatic proxy mode of Adguard.

Now along with this "Allowed IP", DNS, MTU, Addresses and Excluded app should also matter.

In my testing I've found ads to bypass ag filtering when I allow IPv6 address in allowed IP and also when I use 1.1.1.1 in dns field probably because the ad requests passing through wireguard-warp resolves through that dns instead of ag. Currently I've left it blank and sometime I use it with some adblocking dns which stops the ads but leaves the white spaces. All these experimentation keep giving different results depending on several factors like some data may get cached etc as mentioned by wgcf dev here

There's also something special about 0.0.0.0/0 and 0.0.0.0/1 in the way it's treated differently under different situation. I read something about openvpn using 0.0.0.0/1 and 128.0.0.0/1 instead of 0.0.0.0/0 to avoid race conditions with other network control or administration software when connecting or disconnecting from the VPN . It also ensures that the system does not lose network access if the VPN client crashes.

SInce I'm on root and have installed system certificate, the apps using webview like google news also benefit hugely from system certificate and this experimentation shows direct effect in such apps.

Now a major issue apart from the above problems is that the ag vpn mode and automatic proxy mode doesn't exactly behave same has some issues which can be clearly seen in request log. When you'll replicate the above setup, you'll see that when adguard is used in vpn mode, all requests shows as dns requests and when automatic proxy is used the ag dns filtering doesn't properly work. Because of all these I'm unable to come to a conclusion what's exactly happening here and how should I setup.

What I and everybody would ideally want in this situation is to be able to use both of them together without any issues with both of them working at their maximum potential - taking advantage of root, all the protocols and ports/routes connected, whitelisted/excluded at the right places with no data leaks happening whatsoever. Rethinkdns app has dedicated wireguard module but as far as I know it's perhaps based on the slower userspace go backend and also mostly use TCP whereas Warp almost completely works with UDP.

If you want me to experiment with some settings, I'm all ears.

Some useful links :-

• Test whether warp is functional or not 1, 2, 3
• Unofficial wireguard documentation
• Warp announcement article describing some of its working including encryption and compression. WARP utilizes UDP for all of its communications. By default, the UDP port required for WARP is UDP 2408. WARP can fallback to UDP 500, UDP 1701, or UDP 4500.
• A lightweight barebone webview tester to test whether ag is able to block ads and also remove whitespaces or not (so as to completely remove the possibility of browser modifying anything).

[sfionov]

Another report from user with Wireguard in "user mode" + AG in root proxy mode - he experiences problems with IPv6 filtering.

[Rtizer-9]

@sfionov I've found an interesting solution to this whole issue with some caveats. Can we chat on telegram regarding this, you're Sergey Fionov on tg if I'm correct.

[Dondrejohnson5]

• Now the Wireguard app has some interesting settings out of which 75% directly/indirectly should affect traffic routing if you're using it with Ag, one of them is "Allowed IP" which defines the IP ranges for which a peer will route traffic.

Official Cloudflare warp clients have all local address excluded from their routing so ideally we should also be whitelisting them wireguard official app when we're using warp in it. Official wireguard app also has a dedicated option to exclude private IPs.

So I calculated those addresses using this WireGuard AllowedIPs Calculator with some ISP addresses too to let volte/vowifi work just in case.

What were all the allowed IP addresses you got after doing all the above? I tried to replicate what you did but got lost on this step

[Rtizer-9]

@Dondrejohnson5 since my Network uses different addresses for volte/vowifi calling so addresses would be different.

You can simply try 0.0.0.0/0,::/0 at the moment or go to official warp app and use those whitelisted addresses and the addresses your network use for volte/vowifi **(I'm not sure if that's necessary) to input in the above website I've linked and you'll get the addresses.

[Dondrejohnson5]

Ok, I already had those down, so i guess i should be good there..

Now, when you'll use this setup we'll still see ads bypassing Ag even if ag is turned on. Now I'd linked something above which says ipv6 can bypass dns apps so I thought let's stop ipv6 routing through wireguard and I removed ipv6 addresses I got above which I had put in "Allowed IP". Ag team also mentioned specifically about automatic proxy mode having some issue with ipv6 filtering earlier.

And now ads are properly filtered as far as I can test in both vpn and automatic proxy mode of Adguard.

So by that you meant remove ::/0 from Allowed IP or did I misunderstand that as well?

[Rtizer-9]

Yes you're right in concluding that but those calculations might not apply as of now. I'm using another setup helping me use all this but it's not stable as it's causing problem with ipv6 which is unavoidable for my network so I keep losing network.

You can try that if you want.

Making this whole setup work isn't really a big issue , the main issue I'm currently facing is that even if you make ag and Wireguard work together ag will only receive dns queries and not complete urls which will result in way too inferior blocking because out of all the lists you'll be using only domain entries will be applied as a result cosmetic filtering (and almost all blocking except dns level blocking) will stop working completely.

I've found a workaround to this whole issue which will make ag receive those complete urls with app icon and filtering in request log perfectly visible as it should be but then IPv6 stops working.

[Dondrejohnson5]

I have. It works but now YouTube videos buffer much more than before, which kinda defeats the purpose of having warp over wireguard in the first place. With allowed IP ::/0 and 0.0.0.0/0, videos play without a hitch..

Let me know what new solution you've tried

[Rtizer-9]

The buffering in this scenario 99% of the time doesn't happen because warp is slow but because of constant network changes the dns is not found, once dns resolves ... everything is quick.

The allowed ip isn't probably causing that buffering it's mostly the dns resolution.

Can you see YouTube requests with app icon in ag filtering log perfectly with this setup? For me with ag and Wireguard alone with any kind of allowed ip the app icon is not visible and only dns queries can be seen in ag filtering log.

If you force a ag filters update somehow the network changes for a fraction of second then you can see some 5-10 minutes early requests of some app AFTER the current requests ( as in you'll see some 9:50PM requests made by some app when it's already 10PM ..so basically non-chronologically) and then it again switches to those dns queries.

[Dondrejohnson5]

Well yes I see the YouTube logo appear... maybe the problem earlier is I had dns off in Adguard because it caused multiple connection issues, but strangely now, it took a minute at first but I get a stronger connection on YouTube now. I don't know about ipv6 filtering because I'm this is the results I'm stuck with on this page.

[Dondrejohnson5]

In the last hour I've tried a few other things to get ipv6 recognized and it's either put ::/0 back in allowed IP and let ads get away or take it back off and essentially get stuck in ipv4 mode since chrome even can't identify my ipv6 address.. and lose total connection somewhere in the process.. Anyway, keep me posted when you have the ipv6 thing figured out.. if you'd like to pm me on telegram instead

[Rtizer-9]

Let's continue this there.

[Rtizer-9]

@sfionov @ameshkov any update on this? We're still not able to use warp with adguard with cosmetic filtering. Both ag and warp can run together without the stability problems earlier mentioned in the thread but then adguard is not able to capture data of some apps and can only see dns data for them leading to no cosmetic filtering.

Disabling IPv6 completely solves the issue as I mentioned before.

[Rtizer-9]

#5370 solved this which was the sole reason for causing this.

[Rtizer-9]

#5370 solved it for automatic proxy mode but the issue remains still unresolved for even the latest nightly in vpn mode.