AdguardTeam / CoreLibs

Core Adguard libraries
https://adguard.com/
Apache License 2.0
40 stars 7 forks source link

Add support for HTTP/3 (IETF QUIC) ("h3" protocol) #487

Closed sfionov closed 1 year ago

sfionov commented 6 years ago

Add support for HTTP/3 filtering.

TPS commented 6 years ago

@ameshkov Maybe #486?

sfionov commented 6 years ago

@TPS No, IETF QUIC and existing QUIC are different protocols. IETF QUIC is split to QUIC transport (with modified TLS1.3 handshake) and HTTP over QUIC (hq) based on modified HTTP/2. Google QUIC is more solid and have different Alt-Svc/ALPN protocol name.

sfionov commented 5 years ago

nghq seems to be abandoned. Current implementations:

HTTP/3 QUIC transport: https://github.com/ngtcp2/ngtcp2

HTTP/3 application protocol: https://github.com/cloudflare/quiche (Rust, C bindings exist) https://github.com/ngtcp2/nghttp3

DBotThePony commented 4 years ago

nginx is getting official support for QUIC: https://quic.nginx.org/

ameshkov commented 2 years ago

Just a short update on this.

We did make an internal prototype of CoreLibs with HTTP/3 support, but we're not yet happy with how it works. More work is required.

Rtizer-9 commented 2 years ago

Meanwhile can you please add an option to allow quic protocol though instead of blocking everything quic on android.

Option on per app basis will be great. I need it for some testing purpose. I'll be grateful if you can do that before the proper quic/https3 filtering comes in adguard.

sfionov commented 2 years ago

@Rtizer-9 Since Adguard 3.6.5 release, QUIC is not blocked in apps without HTTPS filtering on. (This applies to Local VPN mode only).

If you still have issues, there is also an option in advanced settings with list of app uids for explicit QUIC bypass - pref.quic.bypass.packages

Rtizer-9 commented 2 years ago

Oh great. It escaped my attention. Thanks.

ammnt commented 2 years ago

Any update of this? HTTP3 support for NGINX is coming soon!🎉

szhu25 commented 2 years ago

IETF recently published HTTP/3 as RFC 9114. Can we get any status update or (more distantly) a timeline for HTTP/3 support within AdGuard products?

Thanks!

ryboe commented 2 years ago

I would like to see this prioritized as well. HTTP/3 is a big performance win for mobile. It's a shame to be missing out on the benefits of a UDP-based protocol. If HTTP/3 support gets added as experimental, I'll enable it so hopefully the devs can collect some telemetry.

TPS commented 2 years ago

I'm waiting for this before I re-enable HTTP/3 on my devices across the board. Can't go back to the pre-AG days anymore!

ryboe commented 2 years ago

HTTP/3 is currently 25.5% of website traffic, according to w3tech.com.

TPS commented 2 years ago

But before lets see who will support it.

Looks like mainly properties from Google, Facebook, Microsoft, &c, per above.

@ryboe Thanks for the link. That continuous monitoring info seems very helpful.

zubrRB commented 2 years ago

dnscrypt-proxy version 2.1.2: Support for DoH over HTTP/3 (DoH3, HTTP over QUIC) has been added.

szhu25 commented 1 year ago

It's almost a year since the last comment and the target corelibs label has not been updated for half a year now... Would this be possible released in v1.12?

Asking because now Nginx have experimental HTTP/3 support in their mainline version, so hoping HTTP/3 adoption rates would gradually increase for small website owners.

ameshkov commented 1 year ago

@stevenzhu25 it will be available in AdGuard versions that use CoreLibs v1.12, but behind a low-level flag.

Expect it to arrive to the nightly channel in about a month from now.

sfionov commented 1 year ago

Experimental HTTP/3 support has been implemented in CoreLibs 1.12.

It will be available in the next release cycle of desktop AG products, although it is disabled by default.

It should work ok in Safari and Edge. But at this time it will not work in Chrome unless --origin-to-force-quic-on= is specified with name of site.

It is because Chrome's QUIC certificate verification code does not expect middlebox CA certs, which prevents HTTP/3 from being filtered by default. However, they have a task to unify TLS and QUIC verification (https://crbug.com/848277), so it may change in future.

szhu25 commented 1 year ago

It should work ok in Safari and Edge.

Thanks for the update! Any testing to see if it would work in Firefox?

sfionov commented 1 year ago

@szhu25 Yes, it will work in Firefox.

TPS commented 1 year ago

@ameshkov What are the target nightly version #s for each platform, especially mobile?

ameshkov commented 1 year ago

Not yet planned on exact nightlies of Android, but it will be done in v4.2 indeed.

war59312 commented 1 year ago

See was enabled in v7.15.0-beta-1 for firefox and edge.

No luck here it seems. Enabled the advanced setting for http3 filtering and using Edge on Win11 22H2.

https://quic.nginx.org/quic.html still shows using HTTP and https://cloudflare-quic.com/ still shows using HTTP/2 after many refreshes.

BooBerry commented 1 year ago

I do see some HTTP/3 connections on some sites, like Google, via the developer tools (Network tab, enable the Protocol column).

However, it seems enabling both the Filter HTTP/3 and Use HTTP/3 for DNS-over-HTTPS options causes a lot more AG Service crashes since HTTP/3 support landed in nightly. It can crash multiple times in a row, or even after rebooting the PC with both options enabled. Disabling Use HTTP/3 for DNS-over-HTTPS seems to stabilize it.

ameshkov commented 1 year ago

Note, that it will not work in Chromium-based browsers until this bug is resolved.

BooBerry commented 1 year ago

It does work, kinda, in Firefox. Even though HTTP/3 is used for some things on sites like Google, for others like Facebook and https://cloudflare-quic.com/ and https://http3.is/ it's still using HTTP/2 even though 1) it's enabled in AG's advanced settings and 2) it's enabled in Firefox.

sfionov commented 1 year ago

@BooBerry Can you please send a crash report and say what time report is sent?

About falling back to HTTP/2 - I reproduced it on Facebook and https://cloudflare-quic.com/, thanks.

BooBerry commented 1 year ago

@sfionov Sent a report in, #800123 is the number, Crash happened at 1:09 PM my time, sent in within 2 minutes ago at 1:11 PM my time.

Hopefully that helps. :)

BooBerry commented 1 year ago

Also did some more HTTP/3 testing in Firefox (developer console with the Protocol tab enabled) and it looks like with AG enabled it only uses HTTP/3 for one connection, for the RotateCookies connection. I'm seeing this on https://www.google.com and https://news.google.com at the moment.

Other sites like https://www.twitter.com and https://www.youtube.com and https://outlook.live.com and https://mail.google.com it isn't used at all, whereas without AG (testing with a portable Firefox not filtered by AG but set up exactly the same) I see HTTP/3 being used quite a bit.

Same with https://http3.is where Firefox with AG falls back to HTTP/2 only, and Firefox alone uses HTTP/3.

sfionov commented 1 year ago

@BooBerry A fix is available in Mac version 2.12 beta 1 and Windows version 7.15 nightly 10+ (and will be included into beta 2 soon).

BooBerry commented 1 year ago

For which issue? The crash issue or the HTTP/3 not working on sites in Firefox issue?

sfionov commented 1 year ago

@BooBerry HTTP/3 filtering in Firefox should be fixed.

As of "HTTP/3 for DNS-over-HTTPS", it is known issue that it crashes, it is recommended not to switch on this feature in 7.15 release.

BooBerry commented 1 year ago

Just double checked 7.15 nightly 12 and here's the results.

1) Random AG service crashes still exists when Filter HTTP/3 and Use HTTP/3 for DNS-over-HTTPS are both enabled at the same time. If I disable Use HTTP/3 for DNS-over-HTTPS, no more crashes of the AG service.

2) Just tested Firefox 117.0.1 with it (and a cleared browser cache) and it's still not using HTTP/3 for anything but RotateCookies. I'm going to do a backup of the Firefox profile and first try deleting all cookies then try a clean profile.

sfionov commented 1 year ago

Did you try reloading page twice? Few reloads may be needed to switch to HTTP/3 - after first reload Alt-Svc is cached, and after second it is applied.

BooBerry commented 1 year ago

Yeah, just double checked with a clean profile in Firefox, HTTP/3 isn't being used with AG enabled (and Filter HTTP/3 enabled).

And yes, I reloaded multiple times. It works with AG disabled, of course.

sfionov commented 1 year ago

There was an issue in driver configuration that blocked HTTP/3 from being filtered under Windows. The fix will be included into the next beta.

BooBerry commented 1 year ago

Groovy, I assume it'll be in the next nightly (nightly 17 or newer) too?

TPS commented 1 year ago

Note, that it will not work in Chromium-based browsers until this bug is resolved.

@ameshkov, @sfionov: Just to confirm, the announcement re: this issue @ https://github.com/AdguardTeam/AdguardForAndroid/releases/tag/v4.2-beta-1 really only means Ff-based browsers, correct? Chromium-based are still SOL d/t this bug? Or was there progress somehow, & this issue wasn't updated? I'm still having very similar behavior as @BooBerry detailed previously, so wanted to verify expectations.

BooBerry commented 1 year ago

No, it won't work in Chromium-based browsers right now until Google resolves it, just Firefox (and I think Safari?) right now.

The currently AG for Windows nightly build does work with HTTP/3 in Firefox finally, but as I was expecting it's not any faster than HTTP/2 in my own testing, but it's something. :D

sfionov commented 1 year ago

@TPS MS Edge should work too, it is Chromium-based but have, I believe, another QUIC implementation.

TPS commented 1 year ago

@TPS MS Edge should work too, it is Chromium-based but have, I believe, another QUIC implementation.

@sfionov MSEdge Android is specifically failing. It says it'll accept QUIC on 1st connection (ALPN or 0RTT, IIUC), but all QUIC connections fail in actual use, using test sites as provided above.

Would someone confirm/deny my experience?

BooBerry commented 1 year ago

No-go for Edge here on Windows, unfortunately.

TPS commented 1 year ago

@BooBerry Just to confirm: did you force Enabled the QUIC options under edge://flags? I did so under Android to give it the best chance of success.

BooBerry commented 1 year ago

Of course, I've had the QUIC option force enabled in Edge (and Chrome) the whole time.

sfionov commented 1 year ago

@TPS @BooBerry Tested on my Mac, it doesn't work too. Seems that previous test was incomplete. So it is true, feature is available only on FF and Safari. :(

You may also use --origin-to-force-quic-on=www.google.com:443 Chrome command line option to force quic on your favorite domain in Chrome, but it is not a solution for all sites.

TPS commented 1 year ago

You may also use --origin-to-force-quic-on=www.google.com:443 Chrome command line option to force quic on your favorite domain in Chrome, but it is not a solution for all sites.

@sfionov This causes QUIC to work while on AG? Is this somehow different than the ://flags options as mentioned previously?

sfionov commented 1 year ago

@TPS This option disables certificate verification for that site and forces HTTP/3. So yes, it should work with AG. But if you try to enable it on all sites, every non-HTTP/3 site will be broken.

szhu25 commented 1 year ago

I tested http3 using AdGuard for Android beta with Firefox, and it doesn't seem like http3 is working. Edit: AdGuard for Windows & Android Beta (with filter http3 toggled) & Firefox release (with http3 enabled) does not use HTTP/3 connection on both http3.is & cloudflare-quic.com. If I disable AdGuard, both Android and Windows desktop Firefox would use http3 on cloudflare-quic.com and http3.is.

ghost commented 1 year ago

On iOS Safari 17, h3 is working with https://cloudflare-quic.com, but not https://http3.is/.

image

image

sfionov commented 1 year ago

@jslawler CoreLibs are only used on Android, Mac and Windows.

http3.is is hard to get working even without AdGuard - several reloads are needed, and result is not guaranteed