Patches for the unix build

[mattytrentini]

The unix build appears to have been broken for some time; this doesn't resolve all the issues but makes a start.

[mattytrentini]

I'm building on WSL (Ubuntu 22.04) and have read through #113. I'm aware that there may be some challenges running the built executable (due to the SDL version packaged with 22.04 as well as the differences running under WSL) but it appears that there are some compile/link issues that may have crept in.

Current issue is:

❯ python3 make.py unix DISPLAY=sdl_display INDEV=sdl_pointer
Previous HEAD position was 6ad390fc5 File dialog improvements
HEAD is now at f461d91cd Updated to version 2.30.2 for release
make: Entering directory '/root/code/mattyt-lvgl-micropython/lib/micropython/ports/unix'
Use make V=1 or set BUILD_VERBOSE in your environment to increase build verbosity.
Including User C Module from /root/code/mattyt-lvgl-micropython/ext_mod/lcd_bus
Including User C Module from /root/code/mattyt-lvgl-micropython/ext_mod/lcd_utils
Including User C Module from /root/code/mattyt-lvgl-micropython/ext_mod/lvgl
Updating submodules: lib/mbedtls lib/berkeley-db-1.xx lib/micropython-lib
[snip]
CC build-standard/frozen_content.c
LINK build-standard/micropython
/usr/bin/ld: build-standard/main.o: in function `lcd_bus_init_sdl':
/root/code/mattyt-lvgl-micropython/lib/micropython/ports/unix/modmachine.h:12: multiple definition of `lcd_bus_init_sdl'; build-standard/machine_timer.o:/root/code/mattyt-lvgl-micropython/lib/micropython/ports/unix/modmachine.h:12: first defined here
/usr/bin/ld: build-standard/main.o: in function `lcd_bus_deinit_sdl':
/root/code/mattyt-lvgl-micropython/lib/micropython/ports/unix/modmachine.h:19: multiple definition of `lcd_bus_deinit_sdl'; build-standard/machine_timer.o:/root/code/mattyt-lvgl-micropython/lib/micropython/ports/unix/modmachine.h:19: first defined here
make: Leaving directory '/root/code/mattyt-lvgl-micropython/lib/micropython/ports/unix'
collect2: error: ld returned 1 exit status
make: *** [../../py/mkrules.mk:233: build-standard/micropython] Error 1

I'll keep chipping away but if anyone else has successfully build for unix I'd appreciate any guidance!

[kdschlosser]

OK I fixed that last error you were having. If you can pull the changes into your branch an and push the commits to this PR that should fix that last error you are getting. If you test it and it compiles properly then I will merge your PR.

[kdschlosser]

Was that trailing slash causing a problem. because I am pretty sure it needs to be there in order to work properly. It's a macro and there needs to be a blank line after the macro because it is getting added to an array.

[mattytrentini]

OK, I can confirm that main now builds cleanly! Excellent work. 😄

There does seem to be some sort of problem however; the REPL takes >2 min to start.

Starting the simple 'slider' demo (in the build command examples)) takes 4.5 minutes. Once the app has started, the slider appears very responsive.

During the periods waiting for the REPL or for the app to start, none of the logical CPU's are at maximum. Very odd! This computer is an i7 with 4 'real' and 12 logical cores and 32GB RAM and I'm running on WSL (Ubuntu 24.04) on Windows 10.

In any case, I can close this PR. Perhaps open an issue about that slow starting times?

[kdschlosser]

I am not 100% sure why it is taking so long for your to load. on my local machine basically the same setup as you except a faster processor and more logical cores it loads instantly. Now delay what's so ever.

It would help if I can see the code you are running.

[kdschlosser]

I just cloned the repo and compiled the unix binary.

This is how it runs using the code from the example....

sorry about me fat fingering the import... But you get the idea on how fast it loads.

[mattytrentini]

Building from main (at f0da49d). Just running and exiting the REPL:

> git log -n 1 --oneline
f0da49d (HEAD -> main, origin/main, origin/HEAD) gets timer working for unix and macOS ports.
> python3 make.py unix DISPLAY=sdl_display INDEV=sdl_pointer
[snip]
> chmod +x ./build/lvgl_micropy_unix
> time ./build/lvgl_micropy_unix -c "import machine; machine.soft_reset()"

real    2m10.825s
user    0m0.009s
sys     0m0.001s

[kdschlosser]

try adding the code into a python file. get the repl up and running and then import the file. See if it behaves any differently.

I am not sure as to why you would be having this issue.

what version of WSL are you using?

[mattytrentini]

try adding the code into a python file. get the repl up and running and then import the file. See if it behaves any differently.

That's what I did; it takes 4.5 mins to complete the import sdl_test and the UI to appear.

The REPL - just the REPL - takes 2+ mins to appear.

I am not sure as to why you would be having this issue.

I'm not sure either. "Normal" MicroPython starts instantly:

> time ports/unix/build-standard/micropython -c "import machine; machine.soft_reset()"

real    0m0.004s
user    0m0.000s
sys     0m0.001s

what version of WSL are you using?

> wsl --version
WSL version: 2.3.24.0
Kernel version: 5.15.153.1-2
WSLg version: 1.0.65
MSRDC version: 1.2.5620
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.26100.1-240331-1435.ge-release
Windows version: 10.0.19045.4894

[kdschlosser]

I know there are some issues with Windows 10 and WSL and this might be one of those problems. I am running Windows 11 and WSL 2

Here is a version comparison between WSL1 and WSL2.

https://learn.microsoft.com/en-us/windows/wsl/compare-versions

[mattytrentini]

Here is a version comparison between WSL1 and WSL2.

Just to clarify, I am using WSL2.

Given that I haven't had any obvious performance issues with any other applications, it seems more likely to be something related to lvgl_micropython. Not necessarily lvgl_micropython itself - it could be a dependency or something unusual in my build toolchain. But I presume the runtime process of getting to the REPL is similar to regular (unix) MicroPython...but it must be different somehow!

I'll investigate further...I can start by building with the same toolchain as regular MicroPython (which was built in a container, rather than lvgl_micropython which was built directly on the WSL host).

[mattytrentini]

It's not the toolchain 😞:

> docker run -ti --rm -v $(pwd):/code -w /code gcc:12-bookworm bash -c "python3 make.py unix DISPLAY=sdl_display INDEV=sdl_pointer"
[snip]
LINK build-standard/micropython
text       data     bss     dec     hex filename
3519707   72800   32000 3624507  374e3b build-standard/micropython
make: Leaving directory '/code/lib/micropython/ports/unix'
compiled binary is /code/build/lvgl_micropy_unix

> time ./build/lvgl_micropy_unix -c "import machine; machine.soft_reset()"

real    2m10.684s
user    0m0.010s
sys     0m0.001s

[kdschlosser]

run a printenv. I want to see if you have anything stored in the way of compiler arguments that might be causing this to happen...

also do you have wayland installed?

what version of GCC are you using?

[mattytrentini]

> printenv
SHELL=/bin/bash
WSL2_GUI_APPS_ENABLED=1
WSL_DISTRO_NAME=Ubuntu-20.04
WT_SESSION=dd3a8312-0ab4-49c3-a39b-91eaf6fa7175
EDITOR=vim
NAME=mst-lap
PWD=/root/code/mattyt-lvgl-micropython
LOGNAME=root
IPADDRESS=172.31.192.1
HOME=/root
LANG=C.UTF-8
WSL_INTEROP=/run/WSL/56755_interop
STARSHIP_SHELL=bash
WAYLAND_DISPLAY=wayland-0
LESSCLOSE=/usr/bin/lesspipe %s %s
TERM=xterm-256color
LESSOPEN=| /usr/bin/lesspipe %s
USER=root
DISPLAY=172.31.192.1:0
SHLVL=1
XDG_RUNTIME_DIR=/mnt/wslg/runtime-dir
WSLENV=WT_SESSION:WT_PROFILE_ID:
XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
PATH=/root/.rye/shims:/root/.cargo/bin:/root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/Program Files (x86)/Common Files/Oracle/Java/javapath:/mnt/c/Program Files/Renode/bin:/mnt/c/Program Files/ImageMagick-7.1.0-Q16-HDRI:/mnt/c/windows/system32:/mnt/c/windows:/mnt/c/windows/System32/Wbem:/mnt/c/windows/System32/WindowsPowerShell/v1.0/:/mnt/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common:/mnt/c/Program Files/dotnet/:/mnt/c/Program Files/Gource/cmd:/mnt/c/Program Files/usbipd-win/:/mnt/c/windows/System32/OpenSSH/:/mnt/c/Program Files/Git/cmd:/Docker/host/bin:/mnt/c/Program Files (x86)/PuTTY/:/mnt/c/Users/mst/.poetry/bin:/mnt/c/Users/mst/AppData/Local/Programs/Python/Python310/Scripts/:/mnt/c/Users/mst/AppData/Local/Programs/Python/Python310/:/mnt/c/Users/mst/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/mst/AppData/Local/Programs/Microsoft VS Code/bin:/mnt/c/Users/mst/AppData/Local/Packages/PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0/LocalCache/local-packages/Python310/Scripts:/mnt/c/users/mst/.local/bin:/mnt/c/users/mst/appdata/roaming/python/python310/scripts:/mnt/c/Users/mst/AppData/Roaming/Python/Scripts:/snap/bin:/root/code/gcc-arm-toolchain/gcc-arm-none-eabi-10-2020-q4-major/bin
HOSTTYPE=x86_64
PULSE_SERVER=unix:/mnt/wslg/PulseServer
WT_PROFILE_ID={07b52e3e-de2c-5db4-bd2d-ba144ed6c273}
OLDPWD=/root/code
_=/usr/bin/printenv

Yes, Wayland is installed (see WAYLAND_DISPLAY above). Not sure why it would affect starting the REPL though?

The GCC version in the gcc:12-bookworm container, used for building is 12.4. FWIW my local version is 11.4.

> docker run -ti --rm -v $(pwd):/code -w /code gcc:12-bookworm bash -c "gcc --version"
gcc (GCC) 12.4.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

> gcc --version
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[kdschlosser]

SDL2 is initialized when MicroPython starts and it could be something to do with that.

have you tried placing the example code into a python file named sdl_test.py and place that file in the same directory as the binary. Then just run the binary and from the repl key in import sdl_test...

Try that and see if the problem still exists.

[mattytrentini]

SDL2 is initialized when MicroPython starts and it could be something to do with that.

OK, that might provide a hint...I'll expand the search to maybe a SDL with WSL/WSLg/Ubuntu 22.04 issue.

have you tried placing the example code into a python file named sdl_test.py... Try that and see if the problem still exists.

Yes, same result.

[kdschlosser]

you're guess is as good as mine would be. I have compiled it in a VM running Windows 10 and SDL2 and it is running without any issue. I am not sure what could be doing it environment wise.

[kdschlosser]

do me a favor. run this build command python3 make.py unix clean DISPLAY=sdl_display INDEV=sdl_pointer > output.txt and attached the output.txt file when it is done building. Nothing will be output to the screen when it is building so just be patient and it will finish.

I want to compare your SDL2 build to what mine is. see if there is something that is different.

[mattytrentini]

Attached.

(BTW piping tee to create output.txt also allows the output to screen to be visible ie command | tee output.txt)

output.txt

[kdschlosser]

you didn't add clean to the build command.

[mattytrentini]

Gah, apologies, I was rushing too much; silly mistake! Attached.

output2.txt

[kdschlosser]

ok another thing for you to run..

apt --installed list > applist.txt

I need the applist.txt file.

[kdschlosser]

There are 2 things I can see as possibly causing an issue. The first one is the version of Ubuntu you are running is rather dated.

the second is you do not have all of the packages installed that I have outlined in the README. That being said, some of these packages might not be available for the version of Ubuntu you are running. But if you want to give this a try to see what happens you are more then welcome to..

sudo apt-get install build-essential libffi-dev pkg-config cmake ninja-build gnome-desktop-testing libasound2-dev libpulse-dev libaudio-dev libjack-dev libsndio-dev libx11-dev libxext-dev libxrandr-dev libxcursor-dev libxfixes-dev libxi-dev libxss-dev libxkbcommon-dev libdrm-dev libgbm-dev libgl1-mesa-dev libgles2-mesa-dev libegl1-mesa-dev libdbus-1-dev libibus-1.0-dev libudev-dev fcitx-libs-dev libpipewire-0.3-dev libwayland-dev libdecor-0-dev ccache

The shit thing about using the package manager is the version for the libraries it has are old. They are not updated most times after it has been released to the package manager. Upgrading your version of Ubuntu will provide newer versions of the package because the packages are compiled using the latest versions of them when a new version of Ubuntu comes about. So the problem could be in any of the packages that is listed in that really long lost of them,...

in order for to to have complete control over the build to ensure that everything will work across all versions and flavors of Linux I would need to compile each and every single one of those packages along with any dependencies the package has. That is simply way too much to have to do...

If running the command I pasted above doesn't work I am going to have to be the bringer of bad news and I am going to tell you that you need to get a newer version of Ubuntu installed into WSL..

to upgrade Ubuntu on WSL...

from a Ubuntu terminal

sudo apt update && sudo apt full-upgrade

from a Powershell terminal You might need to alter the distribution name in the second command below.

wsl -l -v
wsl --terminate Ubuntu

Open a new Ubuntu terminal

sudo do-release-upgrade

[mattytrentini]

I installed all the dependencies and tried re-running; unfortunately I observed the same long startup. 😞

Here is the installed list of packages: applist.txt

WSL supports multiple Linux hosts so I'll install the latest Ubuntu in parallel and try rebuilding...

[mattytrentini]

OK! It took a while to install everything but Ubuntu 24.04 seems to run it perfectly, no delays whatsoever! Thanks for persevering @kdschlosser. I'm not sure what the issue is with Ubuntu 22.04 but I'm happy we have found a workable solution.

[kdschlosser]

some hiccup somewhere that is not within my control unless I compile every single requirement for SDL2. That would be a very large amount of additional code I would have to add to do that. So we know the issue is external and not being caused by the binding, that's a good thing to know. so if anyone else has this issue I will know immediately how to solve it. Sorry you had to go through the extra work to get it running. I am going to add to the readme that Ubuntu 22.04 is needed at a minimum.

[kdschlosser]

I believe the things in the PR have been taken care of so this should be able to be closed.

[mattytrentini]

Yes, absolutely; thanks for your help!