MagicStack / uvloop

Ultra fast asyncio event loop.
Apache License 2.0
10.44k stars 548 forks source link

Upgrade libuv to v1.48.0 #600

Closed niklasr22 closed 3 months ago

niklasr22 commented 8 months ago

Upgrades libuv to v1.48.0 which fixes a security vulnerability.

I removed two DNS test cases because they raise an error intended by libuv.

niklasr22 commented 8 months ago

I think the pipeline should be able to pass as it did in the PR in my fork repo. Could someone please trigger a retry?

fantix commented 8 months ago

yeah that error in CI looks like an old flake

tapple-cisco commented 5 months ago

I did some investigating about that venerability. I checked if I could reproduce the ‘truncate after 256 bytes’ venerability. I cannot exploit it if I pass hostname as a string, due to this idna encoding line; uvloop accidentally protects you from the libuv venerability:

>>> payload = f'0x{"0"*246}7f000001.example.com'
>>> payload
'0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007f000001.example.com'

>>> import uvloop, asyncio
>>> async def loop_getaddrinfo(addr, port): return await asyncio.get_running_loop().getaddrinfo(addr, port)
>>> uvloop.run(loop_getaddrinfo(payload, 80))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mfulmer/miniconda3/envs/lm2/lib/python3.9/site-packages/uvloop/__init__.py", line 82, in run
    return loop.run_until_complete(wrapper())
  File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
  File "/home/mfulmer/miniconda3/envs/lm2/lib/python3.9/site-packages/uvloop/__init__.py", line 61, in wrapper
    return await main
  File "<stdin>", line 1, in loop_get_addrinfo
  File "uvloop/loop.pyx", line 1528, in getaddrinfo
  File "uvloop/loop.pyx", line 905, in uvloop.loop.Loop._getaddrinfo
UnicodeError: encoding with 'idna' codec failed (UnicodeError: label too long)

This is a similar error that socket gives:

>>> import socket
>>> socket.getaddrinfo(payload, "80")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mfulmer/miniconda3/envs/lm2/lib/python3.9/socket.py", line 954, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
UnicodeError: encoding with 'idna' codec failed (UnicodeError: label too long)

However, if I pass the hostname as bytes, I can bypass the accidental uvloop protection and exploit libuv:

>>> payload = f'0x{"0"*246}7f000001.example.com'.encode()
>>> payload
b'0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007f000001.example.com'
>>> uvloop.run(loop_getaddrinfo(payload, 80))
[(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 80)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_DGRAM: 2>, 17, '', ('127.0.0.1', 80)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_RAW: 3>, 0, '', ('127.0.0.1', 80))]

socket, however, isn’t fooled:

>>> socket.getaddrinfo(payload, "80")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mfulmer/miniconda3/envs/lm2/lib/python3.9/socket.py", line 954, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

I didn’t know about this "0x7f000001" form of hostnames, but, apparently it’s a thing:

>>> socket.getaddrinfo("0x7f000001", "80")
[(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 80)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_DGRAM: 2>, 17, '', ('127.0.0.1', 80)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_RAW: 3>, 0, '', ('127.0.0.1', 80))]

I can’t find any documentation about why that’s considered a valid hostname. It’s obviously a hex encoding of a 4-byte ipv4 address, but, I’ve never seen it written that way

Anyway, maybe you can turn my investigation into a unit test for the security venerability

tapple-cisco commented 5 months ago

regarding the idna encoding error, there's some discussion of whether that error should be handled a different way in the python standard library or not. Just for reference: https://github.com/python/cpython/issues/77139