Support for libSDL2 - Githubissues

nfeske commented 1 year ago

The libSDL2 and friends (like Mesa) are readily available at genode-world. However, to use the library from a Goa project, using CMake in particular, a few pieces are missing in Goa. Fortunately, @ssumpf happens to have those puzzle pieces in a branch https://github.com/ssumpf/goa/commits/doom3.

@ssumpf I would very much appreciate you upstreaming the libSDL2 support! Maybe even along with a simple example of using it with CMake? I'm asking because the latest version of Hatari relies on SDL2 and uses CMake.

ssumpf commented 1 year ago

@nfeske: Okay, will have a look.

mewmew commented 10 months ago

I'd also love to see libSDL2 support being added to Goa. Especially with the usability improvements of Goa in Genode 23.08. I can't wait to take it out for a spin for some hobby projects : )

@ssumpf thanks for working on this!

nfeske commented 10 months ago

I'm actually offering libSDL2 at my depot: https://depot.genode.org/nfeske/bin/x86_64/sdl2/ (x86) and https://depot.genode.org/nfeske/bin/arm_v8a/sdl2/ (64-bit ARM).

In your Goa project you can refer to my archive in your pkg/archives file using the concrete version nfeske/sdl2/2023-04-27.

It might be worth reviewing the directory https://depot.genode.org/nfeske/bin/x86_64/ for other useful archives, e.g., sdl_image is also there, which is often used by libSDL-based applications.

mewmew commented 10 months ago

That's great! Thanks Norman : )

Hope you've had a lovely summer. Did you do a Hack n' Hike this year?

happy hugs /Robin & Daniel

nfeske commented 10 months ago

Hope you've had a lovely summer. Did you do a Hack n' Hike this year?

I spent my summer vacation completely analog, which was quite refreshing for the mind. Feeling nicely re-energized for the next Genode topics now. :)

We haven't had a Hack'n'Hike this year, but in my opinion we should definitely consider it for spring 2024. Would love to know how the others, e.g., @m-stein or @chelmuth, think about it.

mewmew commented 10 months ago

I spent my summer vacation completely analog, which was quite refreshing for the mind. Feeling nicely re-energized for the next Genode topics now. :)

Glad to hear! I too enjoyed some time in nature this summer. Went trekking in the Rocky Mountains in Canada, a long-lived dream of mine.

It's so great to see what you've managed to do with Genode since the first time we met all these years back. An amazing journey to follow <3

mewmew commented 10 months ago

Hi @nfeske!

I've had a blast experimenting with Goa and Rust. Thanks for publishing your sdl2 package to the depot.

I did run into some issues though with getting sdl2 running on the latest version of Genode. In particular, nfeske/src/sdl2/2023-04-27 relies on the legacy 'main' support which was removed in https://github.com/genodelabs/genode/commit/e2836bf68a009906a2468ea03c58109dda9336a0.

I currently get the following link time error when running programs linked against sdl2 (2023-04-27).

100 MiB RAM and 18997 caps assigned to init
[init -> hello_sdl] Error: LD: symbol not found: 'genode_argc'
[init -> hello_sdl] Error: Uncaught exception of type 'Linker::Not_found'
[init -> hello_sdl] Warning: abort called - thread: ep

chelmuth commented 10 months ago

It seems you have to build the sdl2 archive by yourself as libsdl2 was adapted in genodelabs/genode-world@9d779a3d5311f1b390048c89fa2c3b0fda1d53ff too.

chelmuth commented 10 months ago

We haven't had a Hack'n'Hike this year, but in my opinion we should definitely consider it for spring 2024. Would love to know how the others, e.g., @m-stein or @chelmuth, think about it.

I'm very much for opening a Hack'n'Hike 2024 issue after a first offline brainstorming session.

m-stein commented 10 months ago

I'm very much for opening a Hack'n'Hike 2024 issue after a first offline brainstorming session.

https://github.com/genodelabs/genode/issues/4989

nfeske commented 10 months ago

I did run into some issues though with getting sdl2 running on the latest version of Genode.

That's because Goa is officially tied to the most recent Sculpt version, which is not binary-compatible with the latest Genode version. So if you want to target the latest Genode, you'd have to create the package for libSDL2. That's not hard. You can find the pkg recipes for the src and api at the genode-world repository and use the following command to create it locally:

 genode$ ./tool/depot/create robin/bin/x86_64/sdl2 UPDATE_VERSIONS=1

Just on case you haven't already encountered them, I recommend exploring Goa's depot_dir and versions_from_genode_dir features (as described in goa help config) for targeting the latest Genode version with Goa.

cnuke commented 9 months ago

I spent some time to adapt the SDL2 support based on @ssumpf work and with the referenced commits I am able to build yquake2 as well as chocolate-doom with minor adjustments to the original cmake files (while make use of the provided cmake modules by each project).

nfeske commented 7 months ago

While porting SDL-based software and libraries, I encountered a few limitations of Goa.

For SDL2 and the corresponding utility libraries, the include-search path needs to be extended to include/SDL in addition to the default include/ - same thing as for SDL1. Since the API-specific extension of the include search path is such a recurring pattern, we should better aid it by a utility function to avoid code repetition.
The game Numpty Physics uses a plain Makefile w/o install rule. The game data is used as is from source. This became a problem when creating tar build artifact for this data. As the tar command operates in the build directory, it archived all those nice symlinks pointing to src/. Adding a --dereference to the tar command fixes that.

nfeske commented 7 months ago

@jschlatow would you consider the the commits of my sdl2 branch (https://github.com/nfeske/goa/commits/sdl2) for staging?

jschlatow commented 7 months ago

@nfeske Thanks a lot for the fixes. Merged to staging.

jschlatow commented 7 months ago

@cnuke I lost track of your previously referenced commits. Could you check (whenever time permits) how those relate to the changes introduced with 9fbb283 and 0c88812?

cnuke commented 7 months ago

@jschlatow no problem, I'll look into that tomorrow (I think they complement Norman's commits).

cnuke commented 7 months ago

@jschlatow I cleaned the commits up a bit so that initial cmake support for SDL2 is provided:

99d4537 updates the version define to our current libc version (was part of @ssumpf's commit)
913d2ae moves the system library path from the SDL1 quirk to the global cmake flags
98397bc adds SDL2 support
318593b adds SDL2 mixer support
1b7fbeb adds SDL2 net support
b0fdf48 adds dummy modules needed by YQuake2
b02cc5a adds dummy modules needed by dhewm3

Both commits marked with TMP are required for chocolate-doom but at least the second one changes the integration in a substantial way (absolute paths for the libraries) and are referenced to document the requirement. (I am still looking for a better solution that does not require me to change the cmake files…). I am fine with keeping them on my topic branch for now.

I've noticed that some libraries a referenced multiple time when linking (see log snippet below) but I have not investigated why that is yet.

[yquake2:cmake] /usr/local/genode/tool/23.05/bin/genode-x86-gcc -fPIC -fPIC -O2 -D__GENODE__ -m64 -I/develop/depot/cnuke/api/curl/2023-01-23/include -I/develop/depot/cnuke/api/libc/2023-10-03/include/spec/x86_64 -I/develop/depot/cnuke/api/libc/2023-10-03/include/spec/x86 -I/develop/depot/cnuke/api/libc/2023-10-03/include -I/develop/depot/cnuke/api/mesa/2021-08-28/include -I/develop/depot/cnuke/api/sdl2/2023-10-03/include -I/develop/depot/cnuke/api/zlib/2019-02-25/include -I/usr/local/genode/tool/23.05/lib/gcc/x86_64-pc-elf/12.3.0/include -I/develop/depot/cnuke/api/libc/2023-10-03/include/libc -I/develop/depot/cnuke/api/libc/2023-10-03/include/libc-genode -I/develop/depot/cnuke/api/libc/2023-10-03/include/spec/x86/libc -I/develop/depot/cnuke/api/libc/2023-10-03/include/spec/x86_64/libc -I/develop/depot/cnuke/api/sdl2/2023-10-03/include/SDL2 -nostdinc -D__BSD_VISIBLE -D__FreeBSD__=12 -I/develop/depot/cnuke/api/curl/2023-01-23/include -I/develop/depot/cnuke/api/libc/2023-10-03/include/spec/x86_64 -I/develop/depot/cnuke/api/libc/2023-10-03/include/spec/x86 -I/develop/depot/cnuke/api/libc/2023-10-03/include -I/develop/depot/cnuke/api/mesa/2021-08-28/include -I/develop/depot/cnuke/api/sdl2/2023-10-03/include -I/develop/depot/cnuke/api/zlib/2019-02-25/include -I/usr/local/genode/tool/23.05/lib/gcc/x86_64-pc-elf/12.3.0/include -I/develop/depot/cnuke/api/libc/2023-10-03/include/libc -I/develop/depot/cnuke/api/libc/2023-10-03/include/libc-genode -I/develop/depot/cnuke/api/libc/2023-10-03/include/spec/x86/libc -I/develop/depot/cnuke/api/libc/2023-10-03/include/spec/x86_64/libc -I/develop/depot/cnuke/api/sdl2/2023-10-03/include/SDL2 -std=gnu99 -Wall -fno-strict-aliasing -fwrapv -fvisibility=hidden -Wno-format-truncation -Wno-format-overflow -g -Wl,-gc-sections -Wl,-z -Wl,max-page-size=0x1000 -Wl,--eh-frame-hdr -Wl,-rpath-link=. -Wl,-melf_x86_64 -nostartfiles -nodefaultlibs -lgcc -L/home/jws/develop/goa-projects/yquake2/var/abi/x86_64 -Wl,-shared -Wl,--whole-archive -Wl,-l:ldso_so_support.lib.a -Wl,--no-whole-archive -T /home/jws/develop/goa/share/goa/ld/genode_rel.ld -l:curl.lib.so -l:libm.lib.so -l:libc.lib.so -l:egl.lib.so -l:mesa.lib.so -l:sdl2.lib.so -l:zlib.lib.so -shared -o release/ref_gl3.lib.so CMakeFiles/ref_gl3.dir/src/client/refresh/gl3/gl3_draw.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/gl3/gl3_image.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/gl3/gl3_light.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/gl3/gl3_lightmap.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/gl3/gl3_main.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/gl3/gl3_mesh.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/gl3/gl3_misc.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/gl3/gl3_model.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/gl3/gl3_sdl.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/gl3/gl3_surf.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/gl3/gl3_warp.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/gl3/gl3_shaders.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/gl3/gl3_md2.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/gl3/gl3_sp2.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/gl3/glad/src/glad.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/files/pcx.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/files/stb.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/files/wal.c.obj CMakeFiles/ref_gl3.dir/src/client/refresh/files/pvs.c.obj CMakeFiles/ref_gl3.dir/src/common/shared/shared.c.obj CMakeFiles/ref_gl3.dir/src/common/md4.c.obj CMakeFiles/ref_gl3.dir/src/backends/unix/shared/hunk.c.obj -lgcc -lgcc -lgcc -lm -l:libc.lib.so -Wl,--no-undefined -l:sdl2.lib.so

chelmuth commented 7 months ago

The right-most arguments on this command line are CMakeFiles/ref_gl3.dir/src/backends/unix/shared/hunk.c.obj -lgcc -lgcc -lgcc -lm -l:libc.lib.so -Wl,--no-undefined -l:sdl2.lib.so. Anything right of -lgcc must either also be left of it or not use libgcc symbols. As I don't know where these arguments stem from, I can only cross fingers...

jschlatow commented 7 months ago

@cnuke Thanks for the cleanup, I merged the non-TMP commits to staging. I was also puzzled by the multiple mentions of -lgcc and curious about where these arguments come from and investigated...

The toolchain file (Genode.cmake) is typically parsed multiple times by cmake. Every time, the link_libraries(gcc) command therefore appends an -lgcc. I pushed commit ac412fb to a topic branch, which corrects this. The remaining options are appended by the project-specific CMakeLists.txt. Unfortunately, I was unable to find a way to tell cmake to add the -lgcc at the very end without modifying the CMakeLists.txt

jschlatow commented 7 months ago

@cnuke, @ssumpf, @nfeske I noticed the cmake dummy modules added by 1fbba30 and ac412fb. Since I have been reading into the cmake docs today in another context, I am now wondering whether we should rather move the Find*.cmake files into the corresponding api archives. I believe this would also enable us to remove most (if not all) of the SDL2-related quirks from Goa. Enabling Goa to make use of depot-provided Find*.cmake and *Config.cmake files will simplify Goa users to provide api archives without needing us to update Goa with corresponding quirks. I'd be interested in your opinion on this?

nfeske commented 7 months ago

@jschlatow I like this direction as it gives us a chance to keep the further proliferation of quirks in check.

cnuke commented 7 months ago

@jschlatow good question. I don't have a strong opinion but my first gut reaction is not to “pollute” the api archives with descriptions for external build-systems. From that perspective I would rather update Goa than the archives every time we have to cater for different aspects of an external build-system (or even add support for a new one).

That being said, this assessment mostly stems from my ignorance of cmake (in this particular case). So far I only encountered cmake projects in the goa context where I looked into them in more detail that either brought their own Find*.cmake files (and in doing so provided all the settings their cmake usage requires) or where simply stating the dependency in question is available was enough (see CURL).

I share your sentiment, though, that there is indeed room for improvement but I feel I am lacking all the information to provide reasonable suggestion.

cnuke commented 7 months ago

It was brought to my attention that my comment can be perceived as more harshly as I actually intended. From the perspective that the Genode build-system fades into the background when catering to external build-systems first and foremost providing means to integrate the libraries via means already available (api archives) is a pragmatic solution.

jschlatow commented 6 months ago

I just pushed a few changes to my _api_findcmake branch. With these changes, cmake find modules are searched within the used api archives. I added find modules for the sdl2 and sdl2_* archives as a proof of concept in jschlatow/genode-world@4dd1b70 and published the corresponding archives on my depot (version 2023-12-15). This allows removing most of the cmake-related quirks.

@cnuke: Would you like to give these changes a try? Note that cmake projects must not override the CMAKE_MODULE_PATH variable but rather append to it like this:

set(CMAKE_MODULE_PATH "${CMAKE_MODULE_PATH};${CMAKE_SOURCE_DIR}/cmake")

jschlatow commented 6 months ago

@cnuke I had a brief look at chocolate-doom and noticed that I missed a few things, which I fixed in jschlatow/genode-world@012ed82. I uploaded the new archives as 2023-12-15-b.

cnuke commented 6 months ago

@jschlatow thanks for taking care of that, I'll give your commits a try and report back.

cnuke commented 6 months ago

@jschlatow Alright, it works in general but I had to make some adaptions depending on the host system. In my fairly old Debian 10 VM featuring cmake 3.13.4 using ALIAS for SDL2main in such a manner is not supported apparently. That being said, catering such an old system is bogus anyway.

On the other hand, my other system features cmake 3.27.6 but without commit 9872274 it always picks up definitions from /usr/lib/cmake, which results in the build failing for some targets. I noticed your commit b92f880 but that does not seem to make a difference for me.

Browsing cmake's documentation I was wondering if it makes sence for our use-case to ”explicitly” drive it in cross-compiling mode to prevent any cross-pollination.

jschlatow commented 6 months ago

On the other hand, my other system features cmake 3.27.6 but without commit 9872274 it always picks up definitions from /usr/lib/cmake

That is strange because commit b45cae7 should prevent this if I understood the implementation correctly. I had a similar effect though caused by cmake's caching. Only after removing the build directory, the aforementioned commit was effective for me. Have you tried removing the build directory?

cnuke commented 6 months ago

That is strange because commit b45cae7 should prevent this if I understood the implementation correctly. I had a similar effect though caused by cmake's caching. Only after removing the build directory, the aforementioned commit was effective for me. Have you tried removing the build directory?

Yes, I removed var/build several times while I was trying out different things - I started by specifying --debug-find to cmake and went from there. That's why I stumbled upon the different settings in the documentation and this one made it work for me.

jschlatow commented 6 months ago

Out of curiosity, does it help setting CMAKE_SYSTEM_IGNORE_PREFIX_PATH in addition to CMAKE_IGNORE_PREFIX_PATH?

cnuke commented 6 months ago

Out of curiosity, does it help setting CMAKE_SYSTEM_IGNORE_PREFIX_PATH in addition to CMAKE_IGNORE_PREFIX_PATH?

Also setting it to /usr/ makes no difference, does not work either.

jschlatow commented 6 months ago

Then we should probably disable all the CMAKE_FIND_USE_* options.

cnuke commented 6 months ago

Then we should probably disable all the CMAKE_FIND_USE_* options.

I'm on it.

cnuke commented 6 months ago

Well, the original commit 9872274 that mitigates my problem with one project breaks the linphone-sdk because now the required python interpreter cannot be found anymore for obvious reasons via find_program (after all cmake is now prohibited from checking $PATH to look up binaries).

After further consulting the documentation I see your point that ignoring the prefix should do trick, which leaves my puzzled why cmake picks my host's SDL2* config modules (or rather employs the Config mode to search packages) when $PATH is evaluated.

jschlatow commented 6 months ago

Looking at cmake's implementation, I noticed that the prefixes are compared before generating the search paths patterns. I.e. if your $PATH contains something like /usr/lib/cmake/(bin/sbin), cmake will still search the patterns mentioned here with /usr/lib/cmake as a prefix it is not ignored. Can you double check your $PATH and try setting the latter as an ignored prefix (which will automatically ignore /usr/lib and /usr as well)?

jschlatow commented 6 months ago

@cnuke I must correct myself, adding /usr/lib/cmake to the ignored prefixes does not imply /usr/lib and /usr. Also note that you must add --rebuild to Goa's command line when you test any changes in cmake.tcl.

cnuke commented 6 months ago

Alright, shame on me - I completely forgot that on voidlinux /lib is symlinked to /usr/lib, so cmake will pick up /lib/cmake via / from the truncated PATH entries; commit cf8a6e5 solves the problem for such systems.

Sorry for somewhat wasting your time on that problem.

For completeness sake I updated commit 940e01e but I am somewhat indifferent of including it.

jschlatow commented 6 months ago

Thanks for clearing up this mystery. I merged both commits to staging.

jschlatow commented 5 months ago

All fixes entered the master branch today. I also just pushed an example with commit 4fad395 on staging. It still references my modified api/sdl2 archive, though.

genodelabs / goa

Support for libSDL2 #52