Fix the Windows port

[amirgon]

The Windows port on lv_micropython seems to be broken. Need to fix it and add CI tests

Related:

• https://forum.lvgl.io/t/compile-lv-micropython-v8-for-windows/6672
• https://github.com/lvgl/lv_micropython/pull/21

[embeddedt]

I fixed the original error and got a basic CI setup going, but it now seems to be failing due to the SDL driver referencing POSIX signal APIs. This was added just a few months after the original PR in https://github.com/lvgl/lv_binding_micropython/commit/a2b13da5f36fd388a4026dec5af52617c377396e.

https://github.com/lvgl/lv_micropython/runs/3500291602

[amirgon]

but it now seems to be failing due to the SDL driver referencing POSIX signal APIs. This was added just a few months after the original PR in a2b13da.

True, thanks @embeddedt for reminding me this.

The problem we were trying to solve was how to interrupt the REPL (which is blocking read from stdin) in order to run LVGL event loop while the REPL waits for input.
We cannot run LVGL event loop from another thread because of the assumption that everything runs from the same thread and no locks are needed today. It's also simpler to run all Micropython from a single thread.

I was trying to find out whether there is a way to interrupt REPL on Windows. On Linux, read returns EINTR in case of any signal and we use SIGUSR1 to interrupt it and invoke LVGL event loop on the main thread (possible thanks to PEP 475 implementation). But on windows there is no SIGUSR1 only signals that usually terminate the process which we don't want to use for this.

So the options I can see for now are:

• Add another mechanism to interrupt or timeout the REPL blocking read in the Window port, other than signals.
  It's best to open a PR to upstream Micropython to support this. The original PR achieved this by polling but this was not accepted by Micropython maintainers.
• Not support interactive GUI with REPL on Windows. This means that when REPL is waiting for input the GUI is blocked. I'm not sure this is acceptable because on all other ports this problem doesn't exist.
• Find another Windows specific magical way to interrupt the REPL read and cause it to generate EINTR...?
• Run LVGL event loop from another Micropython thread, handle all required locking, etc.
  Another risk is that LVGL event callbacks are called from the event loop, so the user must also be aware of the fact that his LVGL event handlers might run from a thread other than the main thread and take care of locking on user code... In general I think this approach is not desirable.

[embeddedt]

Looking into a solution for that... in the meantime, we have another problem.

MicroPython v1.16-610-g4da3cba35-dirty on 2021-09-04; win32 version
Use Ctrl-D to exit, Ctrl-E for paste mode
>>> import sys
>>> sys.path.append("lib/lv_bindings/lib")
>>> sys.path.append("lib/lv_bindings/driver/SDL")
>>> import advanced_demo
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "advanced_demo.py", line 17, in <module>
  File "lib/lv_bindings/lib/lv_utils.py", line 45, in <module>
RuntimeError: Missing machine.Timer implementation!

machine.Timer doesn't exist on the Windows port, and lv_timer depends on FFI which appears to be Linux-specific.

[amirgon]

machine.Timer doesn't exist on the Windows port, and lv_timer depends on FFI which appears to be Linux-specific.

Actually the SDL driver still contains the code that runs the event loop. It's enabled by default and can be disabled by setting "auto_refresh" to False (this is done on advanced_demo to show how to use the generic event loop with SDL). So a quick solution can be to rely on that and simply avoid the generic event_loop.

A more complete solution would be to implement lv_timer for Windows (the same way we have Linux specific timer on the bindings) which doesn't rely on FFI.
We can do that without the need to change Micropython core, PRs to upstream etc.

[dotnfc]

machine.Timer for windows port, here is a simple PR: https://github.com/micropython/micropython/pull/8342

for the SDL/modSDL.c here is a simple patch to make the mingw port working

diff --git "a/driver/SDL/modSDL.c" "b/driver/SDL/modSDL.c"
index 2919de9..95a2a86 100644
--- "a/driver/SDL/modSDL.c"
+++ "b/driver/SDL/modSDL.c"
@@ -11,6 +11,10 @@
 #include <emscripten.h>
 #endif

+#ifdef _WIN32
+# define SIGUSR1 16
+#endif // 
+
 /* Defines the LittlevGL tick rate in milliseconds. */
 /* Increasing this value might help with CPU usage at the cost of lower
  * responsiveness. */
@@ -107,6 +111,11 @@ STATIC mp_obj_t mp_init_SDL(size_t n_args, const mp_obj_t *pos_args, mp_map_t *k

     if (args[ARG_auto_refresh].u_bool) {
         mp_thread = pthread_self();
+#ifdef _WIN32
+        signal(SIGINT, handle_sigusr1);
+        signal(SIGTERM, handle_sigusr1);
+        signal(SIGABRT, handle_sigusr1);
+#else
         struct sigaction sa;
         sa.sa_handler = handle_sigusr1;
         sa.sa_flags = 0;
@@ -115,6 +124,7 @@ STATIC mp_obj_t mp_init_SDL(size_t n_args, const mp_obj_t *pos_args, mp_map_t *k
             perror("sigaction");
             exit(1);
         }
+#endif // _WIN32
     }

     return mp_const_none;

[amirgon]

machine.Timer for windows port, here is a simple PR: micropython/micropython#8342

We have an lv_timer implementation for unix. In case your PR don't make it into Micropython, we van consider a Windows specific lv_timer on lv_binding_micropython. (but if possible we prefer that in Python instead of C, with FFI).

for the SDL/modSDL.c here is a simple patch to make the mingw port working

Would you like to open a PR?

[dotnfc]

yes, the ffi should be more flexible. but, it is turned off by default on windows port, not like the unix port.

[amirgon]

but, it is turned off by default on windows port, not like the unix port.

Any idea why?
We can turn it on, on lv_micropython.

[dotnfc]

let's give a try for ffi on lv_mpy win port.

[dotnfc]

Actually the SDL driver still contains the code that runs the event loop. I

hi amirgon, i tried this simple solution your mentioned here, it works():

modify lv_utils.py

1. drop lv_timer check for windows

   #try:
   #    from machine import Timer
   #except:
   #    try:
   #        from lv_timer import Timer
   #    except:
   #        raise RuntimeError("Missing machine.Timer implementation!")

2. init() two lines:

   #self.timer = Timer(timer_id)
   #self.timer.init(mode=Timer.PERIODIC, period=self.delay, callback=self.timer_cb)

so, i think, we may give an empty impl(without ffi call windows timer api) for windows port simply, and make it possible run the lv_micropython of windows port, things like:

# {lv_micropython}\lib\lv_bindings\driver\windows\
class Timer:
    def __init__(self, id, freq):
        pass

    def callback(self, cb):
        pass

    def handler(self, signum):
        pass

[dotnfc]

to check SDL being running here is a simple patch for modSDL.c

 driver/SDL/modSDL.c | 7 +++++++
 lvgl                | 0
 pycparser           | 0
 tests/run.sh        | 0
 4 files changed, 7 insertions(+)

diff --git a/driver/SDL/modSDL.c b/driver/SDL/modSDL.c
index 95a2a86..ff9eaf0 100644
--- a/driver/SDL/modSDL.c
+++ b/driver/SDL/modSDL.c
@@ -136,6 +136,12 @@ STATIC mp_obj_t mp_deinit_SDL()
     return mp_const_none;
 }

+STATIC mp_obj_t mp_in_service_SDL() 
+{
+    return mp_obj_new_bool(monitor_active());
+}
+
+MP_DEFINE_CONST_FUN_OBJ_0(mp_in_service_SDL_obj, mp_in_service_SDL);
 STATIC MP_DEFINE_CONST_FUN_OBJ_KW(mp_init_SDL_obj, 0, mp_init_SDL);
 STATIC MP_DEFINE_CONST_FUN_OBJ_0(mp_deinit_SDL_obj, mp_deinit_SDL);
 STATIC MP_DEFINE_CONST_FUN_OBJ_0(mp_refresh_SDL_obj, mp_refresh_SDL);
@@ -150,6 +156,7 @@ STATIC const mp_rom_map_elem_t SDL_globals_table[] = {
         { MP_ROM_QSTR(MP_QSTR_refresh), MP_ROM_PTR(&mp_refresh_SDL_obj) },
         { MP_ROM_QSTR(MP_QSTR_monitor_flush), MP_ROM_PTR(&PTR_OBJ(monitor_flush))},
         { MP_ROM_QSTR(MP_QSTR_mouse_read), MP_ROM_PTR(&PTR_OBJ(mouse_read))},
+        { MP_ROM_QSTR(MP_QSTR_in_service), MP_ROM_PTR(&mp_in_service_SDL_obj) },
 };

diff --git a/lvgl b/lvgl
--- a/lvgl
+++ b/lvgl
@@ -1 +1 @@
-Subproject commit 4bd1e7e9f7acc5295b65440477e76a048094afbf
+Subproject commit 4bd1e7e9f7acc5295b65440477e76a048094afbf-dirty
diff --git a/pycparser b/pycparser
--- a/pycparser
+++ b/pycparser
@@ -1 +1 @@
-Subproject commit 1706a39e0116dde0b2d1c52d67078a9a0ab4dbe7
+Subproject commit 1706a39e0116dde0b2d1c52d67078a9a0ab4dbe7-dirty
diff --git a/tests/run.sh b/tests/run.sh
old mode 100755
new mode 100644

and test code like this:

if __name__ == '__main__':
   if (sys.platform == "win32"):
       import SDL
       while SDL.in_service():          
          pass

[amirgon]

hi amirgon, i tried this simple solution your mentioned here, it works():

The simple solution with the built-in event loop has some disadvantages: the user cannot customize the event loop, no support for uasyncio etc. All these are supported only with the explicit event loop from lv_utils which requires Timer.

so, i think, we may give an empty impl(without ffi call windows timer api) for windows port simply, and make it possible run the lv_micropython of windows port, things like:

We don't need to import lv_utils at all if we are using the SDL built-in event loop. There is no point since the functionality there will not be used in such case.
Instead, we can change display_driver_utils.py to not import and use the event loop in case of Window. But again, this would limit the Windows user very much so it's not the best solution in my opinion.

[dotnfc]

hi amirgon, we may have 3 options for this:

1. simple 'built-in event loop'
2. lv_timer.Timer build on SDL_AddTimer native mod, like sdl_monitor.c/sdl_mouse.c can this be aceptable, if it were possible - for libffi will import dlfcn-win32 on mingw?
3. lv_timer.Timer build on libffi (call TimerQueueXXX)

BTW.

• when will we use the uasyncio, can you give a use case?
• if we used frozen mpy, how many RAM will be used on ESP32 platform? there are 26 .mpy, 58kb on file size.

  [.]                        [..]                       apa106.mpy                 dht.mpy
  display_driver_utils.mpy   ds18x20.mpy                flashbdev.mpy              fs_driver.mpy
  ili9341.mpy                ili9XXX.mpy                imagetools.mpy             inisetup.mpy
  lv_colors.mpy              lv_spi.mpy                 lv_utils.mpy               neopixel.mpy
  ntptime.mpy                onewire.mpy                sdcard.mpy                 [uasyncio]
  uftpd.mpy                  upip.mpy                   upip_utarfile.mpy          utelnetserver.mpy
  webrepl.mpy                webrepl_setup.mpy          websocket_helper.mpy       xpt2046.mpy
  _boot.mpy

[dotnfc]

another thing, is there any possible to reduce the command line length for gen_mpy.py to produce the qstr? windows command line is limited to 8,192.

this will make build lv_mpy with msbuild.exe possible like micropython official does.

[amirgon]

we may have 3 options for this:

(2) and (3) achieve the same thing in different ways, right? I'm not familiar with SDL_AddTimer or dlfcn-win32 to tell which is better. Are both portable to all Windows versions? Is dlfcn dependent on mingw? We want maximum portability with minimum dependencies.

(1) I've changed display_driver_utils.py to work without event loop if Timer is missing, so this would probably work with Windows now (could you check?)

• when will we use the uasyncio, can you give a use case?

This is useful when there are blocking actions (network or file access, input from user or a simply "sleep") and you want to be able to run multiple co-routines in cooperative multitasking (without multithreading). This is very common on javascript and also used on Python - try to read about async, await and uasyncio.
I have one example to demonstrate LVGL+Micropython+uasyncio.

• if we used frozen mpy, how many RAM will be used on ESP32 platform? there are 26 .mpy, 58kb on file size.

Frozen code uses Flash, not RAM. Are you trying to save RAM or Flash?

is there any possible to reduce the command line length for gen_mpy.py to produce the qstr?

I'm not sure to which command line you refer to.
Could you point me to the code or build script line?

[teaalltr]

Any update on this?