search

eventlet / eventlet

Concurrent networking library for Python
https://eventlet.net
Other
1.24k stars 321 forks source link

Eventlet backdoor loses some lines when pasting in bulk #667

Open AGSPhoenix opened 3 years ago

AGSPhoenix commented 3 years ago

Directly copied out of a backdoor session:

>>> print("1")
print("2")
print("3")
print("4")
print("5")
print("6")
print("7")
print("8")
print("9")
print("10")
print("11")
print("12")
print("13")
print("14")
print("15")1
>>> 3
>>>
15
>>>

Versions (probably "any", but for completeness' sake):

(tmp2) phoenix@debian:~$ uname -a
Linux debian 4.19.0-10-amd64 #1 SMP Debian 4.19.132-1 (2020-07-24) x86_64 GNU/Linux
(tmp2) phoenix@debian:~$ python -V
Python 3.6.11
(tmp2) phoenix@debian:~$ pip freeze
dnspython==1.16.0
eventlet==0.29.1
greenlet==0.4.17
six==1.15.0
(tmp2) phoenix@debian:~$

Turns out code.py discards any pending input after successfully running a chunk of code, something I will restrain myself to describing as an interesting design decision.

Is there a good way eventlet could work around this behavior?

AGSPhoenix commented 3 years ago

And just 10 more minutes of searching brings me to the open Python issue for this... from 10 years ago. No time to investigate the ideas there at the moment, unfortunately.

temoto commented 3 years ago

@AGSPhoenix currently, eventlet backdoor basically patches stdin/stdout to socket and runs stdlib code.InteractiveConsole. Personally, I approve implementing REPLoop in backdoor - it's mostly copy code from stdlib and removing sys.stdin/out patching is a huge win for everybody. I've modified backdoor test in f6770dcdb to verify multi-line input and one can continue to work on top of that branch if you like.

AGSPhoenix commented 3 years ago

Further, less tired reading has revealed that my initial assumptions were incorrect. The buffer cleared by code is the internal buffer where it builds a complete statement, not the buffer of input waiting to be processed. (When will I learn not to submit issues when I should have been asleep an hour ago?)

Interestingly, this doesn't occur with code.interact, which uses the same InteractiveConsole internally. Dug a bit deeper and hacked a logger onto the raw_input method in code, and the input() builtin is somehow skipping over lines when eventlet does its stuff with backdoor, but not when used by code.

Maybe the lost lines are somehow being eaten when eventlet switches stdin around? Maybe this is some weird quirk of how eventlet wraps the socket with file methods?

temoto commented 3 years ago

Maybe the lost lines are somehow being eaten when eventlet switches stdin around?

My guess this is possible.

Maybe this is some weird quirk of how eventlet wraps the socket with file methods?

My guess this is not the case. socket.makefile is used a lot, we'd see more errors of this kind.

It shouldn't be much work to make backdoor interact via explicit files. Sorry, I don't have time for it.

AGSPhoenix commented 3 years ago

All right, an hour of wading through CPython internals has destroyed any interest I have in finding the actual bug. An eventlet-specific copy of InteractiveConsole built for socket operations is likely going to be the least ridiculous solution.

AGSPhoenix commented 3 years ago

Sketched out a PoC: https://github.com/AGSPhoenix/eventlet/commit/852df0aa1af979733c8f5a708c27928917123215

It works (usually?), but is not ready for merge. I have several questions about eventlet's internal API I didn't have time to look into. Might make a good base for a proper solution?

temoto commented 3 years ago

@AGSPhoenix what questions?