search

python / asyncio

asyncio historical repository
https://docs.python.org/3/library/asyncio.html
1.04k stars 177 forks source link

Raise RuntimeError when transport's FD is used with add_reader etc #420

Closed 1st1 closed 7 years ago

1st1 commented 7 years ago

This PR fixes #372.

The idea is that low-level socket APIs should refuse to work with FDs that belong to some Transport.

This PR adds private implementations for add_writer, add_reader, remove_writer, and remove_reader. Transports only call private implementation.

Public implementations of those methods do an extra check on the FD, raising a RuntimeError when the FD belongs to some Transport. loop.sock_sendall, loop.sock_connect, loop.sock_accept and loop.sock_recv only call public methods.

FWIW uvloop has a lot of functional network unittests, all of them pass on asyncio with this patch.

asvetlov commented 7 years ago

After failed tests fixing, of course.

1st1 commented 7 years ago

Thank you Andrew! @gvanrossum would be cool if you can take a quick look over this PR, I want to merge it before beta2.

gvanrossum commented 7 years ago

I'm confused. Where is self._transports ever updated? It seems it's always empty, so the check will never fail. What am I missing?

1st1 commented 7 years ago

@gvanrossum Transports register when they are being constructed: https://github.com/python/asyncio/pull/420/files#diff-ddadd6cf041d620407e07c7c4f1b0149R572

gvanrossum commented 7 years ago

I see. What about transports that don't inherit from _SelectorTransport? I guess those don't get this error when misused in debug mode but otherwise no harm is done right?

If that's so I think this is fine for b2.

asvetlov commented 7 years ago

Random thought. Let's assume I want to make my own transport for some reason. Honestly I did it only in aiozmq (ZMQ sockets are very different from regular ones, you know) only but maybe somebody will need it for TCP sockets for some reason. Who knows, in aiohttp we have own StreamReader for working with HTTP chunks etc.

It would be cool to have loop methods for fd protecting by protect/unprotect calls. Not in 3.6, sure -- but maybe in 3.7?

1st1 commented 7 years ago

@gvanrossum I think we cover all transports except SubprocessTransport, but those don't expose the socket via get_extra_info. BTW, the check is always enabled, even in production mode (keeping a WeakValueDict for transports is cheap).

Honestly I did it only in aiozmq (ZMQ sockets are very different from regular ones, you know) only but maybe somebody will need it for TCP sockets for some reason.

@asvetlov Let's think about that after 3.6 :)

asvetlov commented 7 years ago

@asvetlov Let's think about that after 3.6 :)

Agree

1st1 commented 7 years ago

Committed by hand, closing this PR now.