Open pmp-p opened 9 months ago
hi, is this issue related to pyTermTk? if you need some help I already overcome this issue with pyodide using a proxy. (i.e. Sandbox) or you can use the pyTermTk input driver.
If this is related to pyTermTk, (i.e. trying to run it under pygame-web) are you able to provide an example or the instructions to reproduce the issue? I didn't spent much time on it but It is not clear how to run a custom python project using wasi/Emscriptem.
I tried your input routine using pygbag but I cannot get any keypresses from stdin how is the stdin/out handled? I amprinting in the stdout a vt100 ansi sequence, is your default screen compatible? did textual, ncurses2 worked out of the box?
sorry i did not think you would try directly on pygbag - i'm honoured - but the select.select patch is still brewing on my pc and is triggered for now by the first termios.tcsetattr() call that is expected to switch the terminal in raw mode. it's very crude but does the job until patching emscripten is worthwhile.
__select = select.select
def patch_select(rlist, wlist, xlist, timeout=None, /):
global __select
# stdin
if rlist[0] == 0:
return [embed.stdin_select()]
return __select(rlist, wlist, xlist, timeout)
select.select = patch_select
def patch_os_read(fd, sz):
return embed.os_read()
os.read = patch_os_read
the embed module from main.c manage to store all keypressed/mouse events in a c-string that happen beetween two frame call ( either from xterm.js or from a replay ). stdin_select just returns that c-string lentgh while os_read returns a suitable pyobject and truncate the c-string at the same time preventing further read until next frame
fixing emscripten select would allow pyodide to behave normally regarding TTYs. it's not a problem to implement tty ioctl on js side since both pyodide-micropython can all access js it is just a matter of having enough people that agree on how to do it.
For now i'm dumping everything that can/should be fixed here https://github.com/pygame-web/platform_wasm/blob/main/platform_wasm/todo.py you are very welcome to give input, it has been years i did not touch ansi
nb: but in the end it is not specific to web, and should work embedded in async any program too at least on posix OSes.
Please just provide a simple normal event loop, ie without threads and too much OOP :
async def async_mainloop():
while True:
ttydata = "" # os.read(fd)
process_events(ttydata)
paint_term()
await asyncio.sleep(0)
asyncio.run( async_mainloop() )
Not only it would solve wasm/wasi but should also serve for offscreen rendering and input replay tests.
what you mean about without too much OOP? The entire pyTermTk design and structure is based on OOP My library is already compatible with pyodide that is a Wasm impllementation of CPython. If you can show me an example or the steps you are trying to run on your framework I can try to investigate a solution, i.e. With a driver for your use case. I cannot rewrite the core routine just on those hints if I cannot test it.
My library is already compatible with pyodide that is a Wasm impllementation of CPython.
I disagree you are using pyodide proxies : they are not part of cpython. They are a one of possible implementations of interacting with a js host and there are others bridges of that kind and other compilers than emscripten.
So i'm just hoping for a very simple and generic event loop that :
output a stream of utf-8 as output.
Leaving up to the platform to implementing read and write on a real (or not) local (or not) tty.
Your general oop toolkit design is very clean, but imho oop/threading add no value to the core event loop itself because they hide too much of the I/O control flow and its implementation for outside viewer.
i don't have a framework at all, i only use cpython-wasm with four file descriptors instead of the standard three ( in short ALTBUF has its own non blocking fd for input )
The select+os.read above is just common mockup for classic unix terminal handling which seems to suit other terminal toolkit but that should not even be required for a serial terminal (tx/rx no ioctl).
regarding pyodide although i'm a contributor i am (was?) only interested in the module build system and the cpython emscripten support (which has been dropped for cpython 3.13 this month).
I have no intention of diving in pyodide specifics and no one should have to when sticking to POSIX ( async, no threads, standards FD).
i don't think you have to rewrite anything , just document the pyodide (driver input / render step / driver output) control flow so a simpler driver (eg based on async or requestAnimationFrame ) can be derived from it.
Are you able to provide me an interface with those input/output stream I can test? The stdin/out used in Linux/Windows/Pyodide are different, for this reason I had to create a driver for each and a proxy for pyodide since I was not able to find an out of the box solution.
About pyodide, its support for threading and I guess also asyncio is limited/not working, Inits implementation,
I used a single thread and any extra TTkThreading objects are emulated through the built-in Javascript Timeout routines.
I had also to rewrite the input routine since it doesn't provide a proper tty device. So, any input stream coming from xterm.js is pushed straight to the input handler of pyTermTk and any push to the stdout is instead sent to the xterm.js there is no threading involved.
Any keypress is captured by:
this.ttk_input = this.namespace.get("ttk_input");
this.term.onData((d, evt) => { this.ttk_input(d) })
and sent to pyTermKk through:
def ttk_input(val):
kevt,mevt,paste = TTkInput.key_process(val)
if kevt or mevt:
TTkInput.inputEvent.emit(kevt, mevt)
if paste:
TTkInput.pasteEvent.emit(paste)
It is not ideal because it skip an extra threaded routine I use to clear the input buffer, but for my needed was ok.
cpython-wasm behaves the same as linux/mac/posix, the same async code is valid on desktop and also supports generators based greenthreads
code: https://github.com/pygame-web/showroom/blob/main/src/test_vt100.py
test: https://pygame-web.github.io/showroom/pythongit.html#src/test_vt100.py
Are you able to provide me some instructions I can use to try my library on your framework? If I can detect that I am running on your environment I can make a driver optimised for it (as it happen for pyodide or windows), I don't want to change the linux core routine. How are you handling the tty/stdin/out?
again, there's no framework it's normal posix cpython stdin is handled here https://github.com/pygame-web/showroom/blob/c0a07cb8a9bf8416eb31937c4e4e12dab1b090f1/src/test_vt100.py#L115
you don't have a linux or mac to run that code ? if not try WSL2
I am using linux.
then the same code will work.
no, because it already runs flawlessly on linux and I don't want to change the core routine. but if I can detect that I am running on wasm I can make a driver.
I noticed that when I was working on pyodide the os type was emscriptem is it the same for you?
the normal way to detect wasi or emscripten is testing sys.platform it is important to handle both right now, as wasi is the only official wasm platform for cpython git
it is ok, as long I can detect it I can make the driver because the input routine won't be the only one that is going to change, there are many blocking threads and timers all around the code. how can I run my demo on your environment?
python3 -m pip install pygbag
name your demo main.py next to the https://github.com/ceccopierangiolieugenio/pyTermTk/tree/main/TermTk folder and you can use python3 -m pygbag --ume_block 0 main.py
or python3 -m pygbag --PYBUILD 3.12 --git --ume_block 0 main.py
for more recent and it will package main.py + pyTermTk for the browser and provide a xterm console and then you can go to http://localhost:8000?-i
beware of blocking threads they can lock up browser tabs for good if you ever forget to add a yield on vsync.
I made a simple python script I wanted to try:
import platform
def main():
print("Eugenio")
print(f"Platform {platform.system()=}")
if __name__ == "__main__":
main()
if I try to run it using:
python -m pygbag --PYBUILD 3.12 --git --ume_block 0 main.py
I get this error:
self.path='/pygbag/0.0/cpython312/main.wasm' path='/home/one/github/Varie/pyTermTk/tmp/build/web/pygbag/0.0/cpython312/main.wasm'
ERROR 404: https://pygame-web.github.io/pygbag/0.0/pythonrc.py
127.0.0.1 - - [26/Jan/2024 17:32:54] code 404, message File not found
127.0.0.1 - - [26/Jan/2024 17:32:54] "GET //pygbag/0.0/pythonrc.py HTTP/1.1" 404 -
ERROR 404: https://pygame-web.github.io/pygbag/0.0/cpython312/main.wasm
127.0.0.1 - - [26/Jan/2024 17:33:18] "GET //pygbag/0.0/cpython312/main.wasm HTTP/1.1" 200 -
----------------------------------------
Exception occurred during processing of request from ('127.0.0.1', 58968)
Traceback (most recent call last):
File "/usr/lib/python3.10/socketserver.py", line 683, in process_request_thread
self.finish_request(request, client_address)
File "/usr/lib/python3.10/socketserver.py", line 360, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python3.10/http/server.py", line 668, in __init__
super().__init__(*args, **kwargs)
File "/usr/lib/python3.10/socketserver.py", line 747, in __init__
self.handle()
File "/usr/lib/python3.10/http/server.py", line 433, in handle
self.handle_one_request()
File "/usr/lib/python3.10/http/server.py", line 421, in handle_one_request
method()
File "/home/one/.venv.python/lib/python3.10/site-packages/pygbag/testserver.py", line 73, in do_GET
f = self.send_head()
File "/home/one/.venv.python/lib/python3.10/site-packages/pygbag/testserver.py", line 143, in send_head
with h_cache.open() as fh:
File "/usr/lib/python3.10/pathlib.py", line 1119, in open
return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: '/home/one/github/Varie/pyTermTk/tmp/build/web-cache/e9594fa69936d600b1002b61e034ff4f.head'
----------------------------------------
self.path='/pygbag/0.0/cpython312/main.wasm' path='/home/one/github/Varie/pyTermTk/tmp/build/web/pygbag/0.0/cpython312/main.wasm'
ERROR 404: https://pygame-web.github.io/pygbag/0.0/cpython312/main.wasm
127.0.0.1 - - [26/Jan/2024 17:33:25] "GET //pygbag/0.0/cpython312/main.wasm HTTP/1.1" 200 -
----------------------------------------
Exception occurred during processing of request from ('127.0.0.1', 58384)
Traceback (most recent call last):
File "/usr/lib/python3.10/socketserver.py", line 683, in process_request_thread
self.finish_request(request, client_address)
File "/usr/lib/python3.10/socketserver.py", line 360, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python3.10/http/server.py", line 668, in __init__
super().__init__(*args, **kwargs)
File "/usr/lib/python3.10/socketserver.py", line 747, in __init__
self.handle()
File "/usr/lib/python3.10/http/server.py", line 433, in handle
self.handle_one_request()
File "/usr/lib/python3.10/http/server.py", line 421, in handle_one_request
method()
File "/home/one/.venv.python/lib/python3.10/site-packages/pygbag/testserver.py", line 73, in do_GET
f = self.send_head()
File "/home/one/.venv.python/lib/python3.10/site-packages/pygbag/testserver.py", line 143, in send_head
with h_cache.open() as fh:
File "/usr/lib/python3.10/pathlib.py", line 1119, in open
return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: '/home/one/github/Varie/pyTermTk/tmp/build/web-cache/e9594fa69936d600b1002b61e034ff4f.head'
----------------------------------------
That's very weird those 404 are about files that are supposed to be on the github cdn ( and i can access them ) can you access this one https://pygame-web.github.io/pygbag/0.0/cpython312/main.js ?
yes, now it works, the network I was using is not very stable. Thanks. I will check this soon, I've a busy WE
eg based on select.select + os.read ( works fine already with nurses2/batgrl and Textual
for cases where
sys.platform in ('emscripten','wasi')
( but not pyodide)eg typical render task could look like :