HTTP/2-only websites can't be tested

[baknu]

Currently Internet.nl needs webservers to support HTTP/1.1 to be able to test them. If a webserver only supports HTTP/2 (and/or HTTP/3) Internet.nl is not able to test for (1) IPv4/IPv6 similarity, (2) HTTPS and (3) Security Options. This was mentioned by a user in a private message via Mastodon on January 17th, 2024.

Our current HTTP client Requests only supports HTTP/1.1. Apparently support for HTTP/2 in Requests depends on urllib3: https://github.com/urllib3/urllib3/milestone/10

Currently the only mature HTTP client in Python that also does HTTP/2 seems to be https://www.python-httpx.org/

Another option would be to use cURL and Python bindings (PycURL).

For a comparison of HTTP clients: https://www.scrapingbee.com/blog/best-python-http-clients/

BTW note that adding a test for HTTP/2 and HTTP/3 is addressed in a separate issue: https://github.com/internetstandards/Internet.nl/issues/160

[Ousret]

Sorry to barge in, It could be worthy to look into Niquests.

Some perks:

• HTTP/2, and HTTP/3 acquired with no effort.
  • https://niquests.readthedocs.io/en/latest/user/advanced.html#specify-http-3-capable-endpoint-preemptively
  • https://niquests.readthedocs.io/en/latest/user/advanced.html#disable-http-2-and-or-http-3
  • https://niquests.readthedocs.io/en/latest/user/quickstart.html#http-3-over-quic
• IPv6, IPv4 control
  • https://niquests.readthedocs.io/en/latest/user/advanced.html#disable-either-ipv4-or-ipv6
• Security status/parameters used accessible
  • https://niquests.readthedocs.io/en/latest/user/advanced.html#ocsp-or-certificate-revocation
  • https://niquests.readthedocs.io/en/latest/user/advanced.html#event-hooks
• Requests backward compatible

And more.