Windows build with embedded GLib

[oakes]

What is the state of the project on windows? I tried building something based on example.c but i'm getting a SIGSEGV when calling chafa_symbol_map_new. In gdb all it tells me is:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffeca733416 in ntdll!RtlWaitOnAddress () from C:\WINDOWS\SYSTEM32\ntdll.dll

[hpjansson]

Good question. The state is unknown, since I haven't tested it, but I've planned for Windows support to be easily achievable.

chafa_symbol_map_new calls chafa_init, which relies on g_once to serialize the the global init. g_once uses the __atomic_load_n intrinsic or a mutex to achieve this, and WaitOnAddress is a serialization function, so I'm suspicious it's crashing here:

https://github.com/hpjansson/chafa/blob/6fc7358268c33ac426cd348541b41d4fe5ee5618/chafa/chafa-features.c#L90

That particular code looks good, though, so there's likely something else going on. My Windows porting-fu is a little out of date, but a typical cause of grief is mixing DLLs from different build envs, resulting in incompatible bitfield handing, linking with the wrong runtime, etc. So I'd be wary of that.

What does your build env look like - MSVC, MinGW? In particular, how is GLib built and linked?

If you want to test if the issue is related to GOnce, and you're not calling into Chafa from different threads, then you should be able to replace chafa_init with something like the following:

void
chafa_init (void)
{
    static int once = 0;
    if (!once) { init_once (NULL); once = 1; }
}

It wouldn't be a permanent solution, though.

Useful links: Documentation for GOnce and advice on Windows porting from the GLib authors.

[oakes]

I'm using gcc 9.3 installed via scoop. I'm building glib statically, mostly the same as the setup in this repo like we discussed a few months ago. I tried the chafa_init change but it seems to have the same behavior. I'll try to continue debugging; normally i'd reach for valgrind but i can't run that on windows.

[hpjansson]

Big fan of valgrind, it's great when you can use it. Not sure if it's relevant, but building with -fno-omit-frame-pointer can improve stack traces. Maybe see if running it single-threaded improves things. Could be an issue with the Windows glibconfig.h or config.h.

Let me know if you get stuck, I'll think about some alternatives in the meantime.

[hpjansson]

I compared the glibconfig.h and GLib's generated config.h from the working MinGW build to the ones in your repo. They are pretty much identical, so the root cause probably lies elsewhere. Did you try with -fsanitize=address,undefined?

I guess one stumbling block is that we still don't have a good stack trace. Can Scoop be set up with debug symbols etc?

[hpjansson]

Well, I installed Scoop on Win 10 and was able to build the stack dynamically there. Works ok. Building on Windows is kind of an ordeal, though. Cross compilation is much easier :-)

A couple of notes so I don't forget:

• Did this (probably left out a few): scoop install gcc gdb binutils coreutils busybox dd python pkg-config git make meson
• Downloaded source for zlib, glib, freetype, chafa.
• From PowerShell, entered bash.
• Built glib with meson build -Dprefix=/usr/local && ninja -vC build && ninja -vC build install
• Built zlib with configure --prefix=/usr/local
• Built freetype with PKG_CONFIG_PATH=/usr/local/lib/pkgconfig meson build -Dprefix=/usr/local && ninja -vC build && ninja -vC build install
• Chafa was a bit more work:
  • Apply patches from https://pastebin.com/raw/zPU8x8Tv and https://pastebin.com/raw/P3ZjLRiS . Credit to this comment by @longnguyen2004.
  • Configured with PKG_CONFIG=pkg-config.exe PKG_CONFIG_PATH=/usr/local/lib/pkgconfig CC=gcc.exe AR=ar.exe SED=sed.exe GREP=grep.exe LN=ln.exe OBJDUMP=objdump.exe NM=nm.exe DD=dd.exe ./configure --build=x86_64-w64-mingw32
  • Built with make LDFLAGS=-no-undefined && make install
  • The installed chafa.exe seemed to be incorrectly linked, but the one from chafa/tools/.libs/chafa.exe worked ok when run from the same dir as the DLLs.

This is all pretty ad-hoc and shouldn't be taken as some kind of definitive set of instructions :-) I wonder why Chafa was so particularly hard to build. Maybe my packaging autotools are too old. Anyway, another good argument for moving to meson.

Static build (./configure --disable-shared --enable-static) also worked ok, and no need to go into .libs for the exe on that one.

The next step would be to try building your repo with the embedded glib. I think I'm getting debug symbols with line offsets etc, so it should be interesting.

[longnguyen2004]

Oh hello there, it's great that you find my comment helpful, although that was more of a (failed) experiment than a proper build guide, so you really shouldn't follow that.

For building autotools based project, you can either cross compile from WSL2, or use MSYS2. Either of them will provide a Unix-style development environment (or you can just port the build system to CMake, which is highly recommended ☺). I'd suggest going with MSYS2, since they have prebuilt packages for zlib, glib and freetype, which saves some time building it from scratch. Either way, the build process should be pretty much the same as with Linux.

If you go the cross-compile route however, Wine is required, since meson needs to run tests on the host arch.

[hpjansson]

For sure, MSYS2 is the easier route, but @oakes was using Scoop and getting segvs with the resulting binaries, and I don't see why it should not build with Scoop, so we're trying to get to the bottom of it. And also it's interesting to see how Autotools (don't) deal with modern marginal environments :-) The diffs you posted were very helpful in that regard, thanks a ton.

[longnguyen2004]

That diff was an experiment to replace MSYS2 with BusyBox and native Windows port of various Unix tools. However it became too difficult to maintain, and the lack of other Unix tools (such as a Unix-style Perl, used by OpenSSL) ultimately scrapped it. You can still use it to build simple autotools projects, but using MSYS2 or WSL2 is the best way to go (or just port it to CMake, you'll make me happy :)

Also, the SIGSEGV crash might be due to different CRT being mixed together, as you've said. Make sure to not mix toolchains!

[longnguyen2004]

Some more info: gcc and nuwen-mingw is nuwen's mingw-w64 distro gcc-msys and gcc-ucrt is MSYS2, linked with msvcrt and ucrt respectively (use gcc-msys to have the best chance of working code) gcc-llvm and gcc-llvm-ucrt is WinLibs, linked with msvcrt and ucrt respectively gcc45 is old MinGW, don't use it

[hpjansson]

@longnguyen2004 Didn't know about nuwen's distro. Awesome, that's a lot of options :-)

@oakes Found your problem.

It's crashing in thread code because on Win32, GLib needs to initialize the thread subsystem on startup. This is done from its DllMain(), which won't run when linking statically. See glib/glib-init.c. It's fixable by preferentially using a gcc constructor.

Suggested changes based on your fork:

diff --git a/build_windows.sh b/build_windows.sh
new file mode 100644
index 0000000..a6d8171
--- /dev/null
+++ b/build_windows.sh
@@ -0,0 +1,139 @@
+#!/bin/bash
+
+CC="${CC:-cc}"
+
+${CC} \
+  -Ichafa \
+  -Ichafa/internal \
+  -Ichafagen \
+  -Wno-deprecated-declarations \
+  -Wno-macro-redefined \
+  tests/example.c \
+  chafa/chafa-canvas-config.c \
+  chafa/chafa-canvas.c \
+  chafa/chafa-features.c \
+  chafa/chafa-symbol-map.c \
+  chafa/chafa-term-db.c \
+  chafa/chafa-term-info.c \
+  chafa/chafa-util.c \
+  chafa/internal/chafa-base64.c \
+  chafa/internal/chafa-batch.c \
+  chafa/internal/chafa-canvas-printer.c \
+  chafa/internal/chafa-color-hash.c \
+  chafa/internal/chafa-color-table.c \
+  chafa/internal/chafa-color.c \
+  chafa/internal/chafa-dither.c \
+  chafa/internal/chafa-indexed-image.c \
+  chafa/internal/chafa-iterm2-canvas.c \
+  chafa/internal/chafa-kitty-canvas.c \
+  chafa/internal/chafa-palette.c \
+  chafa/internal/chafa-pca.c \
+  chafa/internal/chafa-pixops.c \
+  chafa/internal/chafa-sixel-canvas.c \
+  chafa/internal/chafa-string-util.c \
+  chafa/internal/chafa-symbols.c \
+  chafa/internal/chafa-work-cell.c \
+  chafa/internal/smolscale/smolscale.c \
+  -Iglib \
+  -Iglib/glib \
+  -Iglib/glib/gnulib \
+  -Iglib/glib-windows \
+  -lm \
+  -mthreads \
+  -DGLIB_COMPILATION -DLIBDIR \
+  glib/glib/gmessages.c \
+  glib/glib/garcbox.c \
+  glib/glib/garray.c \
+  glib/glib/gasyncqueue.c \
+  glib/glib/gbacktrace.c \
+  glib/glib/gbitlock.c \
+  glib/glib/gbytes.c \
+  glib/glib/gcharset.c \
+  glib/glib/gconvert.c \
+  glib/glib/gdir.c \
+  glib/glib/genviron.c \
+  glib/glib/gerror.c \
+  glib/glib/gfileutils.c \
+  glib/glib/ggettext.c \
+  glib/glib/ghash.c \
+  glib/glib/ghostutils.c \
+  glib/glib/giochannel.c \
+  glib/glib/giowin32.c \
+  glib/glib/glib-init.c \
+  glib/glib/gwin32.c \
+  glib/glib/glist.c \
+  glib/glib/gmain.c \
+  glib/glib/gmem.c \
+  glib/glib/goption.c \
+  glib/glib/gpattern.c \
+  glib/glib/gpoll.c \
+  glib/glib/gprintf.c \
+  glib/glib/gqsort.c \
+  glib/glib/gquark.c \
+  glib/glib/gqueue.c \
+  glib/glib/grand.c \
+  glib/glib/grcbox.c \
+  glib/glib/grefcount.c \
+  glib/glib/gshell.c \
+  glib/glib/gslice.c \
+  glib/glib/gslist.c \
+  glib/glib/gspawn-win32.c \
+  glib/glib/gstdio.c \
+  glib/glib/gstrfuncs.c \
+  glib/glib/gstring.c \
+  glib/glib/gtestutils.c \
+  glib/glib/gthread-win32.c \
+  glib/glib/gthread.c \
+  glib/glib/gthreadpool.c \
+  glib/glib/gtimer.c \
+  glib/glib/gtranslit.c \
+  glib/glib/gtrashstack.c \
+  glib/glib/gunidecomp.c \
+  glib/glib/guniprop.c \
+  glib/glib/guri.c \
+  glib/glib/gutf8.c \
+  glib/glib/gutils.c \
+  glib/glib/gvariant-core.c \
+  glib/glib/gvariant-parser.c \
+  glib/glib/gvariant-serialiser.c \
+  glib/glib/gvariant.c \
+  glib/glib/gvarianttype.c \
+  glib/glib/gvarianttypeinfo.c \
+  glib/glib/gwakeup.c \
+  glib/glib/libcharset/localcharset.c \
+  glib/glib/gnulib/asnprintf.c \
+  glib/glib/gnulib/frexp.c \
+  glib/glib/gnulib/frexpl.c \
+  glib/glib/gnulib/isnand.c \
+  glib/glib/gnulib/isnanf.c \
+  glib/glib/gnulib/isnanl.c \
+  glib/glib/gnulib/printf-args.c \
+  glib/glib/gnulib/printf-frexp.c \
+  glib/glib/gnulib/printf-frexpl.c \
+  glib/glib/gnulib/printf-parse.c \
+  glib/glib/gnulib/printf.c \
+  glib/glib/gnulib/signbitd.c \
+  glib/glib/gnulib/signbitf.c \
+  glib/glib/gnulib/signbitl.c \
+  glib/glib/gnulib/vasnprintf.c \
+  glib/glib/gnulib/xsize.c \
+  -D_FILE_OFFSET_BITS=64 \
+  -D_GNU_SOURCE \
+  -mms-bitfields \
+  -Wl,--allow-shlib-undefined \
+  -Wl,--subsystem,console \
+  -lole32 \
+  -loleaut32 \
+  -luuid \
+  -lws2_32 \
+  -lkernel32 \
+  -luser32 \
+  -lgdi32 \
+  -lwinspool \
+  -lshell32 \
+  -lcomdlg32 \
+  -ladvapi32 \
+  -ggdb \
+  -fno-omit-frame-pointer \
+  -o chafawin && echo "Built chafawin"
+
diff --git a/glib/glib-windows/generated_config.h b/glib/glib-windows/generated_config.h
index 83f7bf5..7b0eb6a 100644
--- a/glib/glib-windows/generated_config.h
+++ b/glib/glib-windows/generated_config.h
@@ -19,7 +19,7 @@

 #define DLL_EXPORT

-#define ENABLE_NLS 1
+#undef ENABLE_NLS

 #define EXEEXT ".exe"

diff --git a/glib/glib-windows/glibconfig.h b/glib/glib-windows/glibconfig.h
index 9d2e3b0..4c32c4c 100644
--- a/glib/glib-windows/glibconfig.h
+++ b/glib/glib-windows/glibconfig.h
@@ -18,8 +18,8 @@
  */
 #undef GLIB_USING_SYSTEM_PRINTF

-/* #undef GLIB_STATIC_COMPILATION */
-/* #undef GOBJECT_STATIC_COMPILATION */
+#define GLIB_STATIC_COMPILATION
+#define GOBJECT_STATIC_COMPILATION

 G_BEGIN_DECLS

diff --git a/glib/glib/ggettext.c b/glib/glib/ggettext.c
index 3360e64..1500a29 100644
--- a/glib/glib/ggettext.c
+++ b/glib/glib/ggettext.c
@@ -40,7 +40,10 @@

 #include <string.h>
 #include <locale.h>
-#include <libintl.h>
+
+#ifdef ENABLE_NLS
+# include <libintl.h>
+#endif

 #ifdef G_OS_WIN32

diff --git a/glib/glib/glib-init.c b/glib/glib/glib-init.c
index 982906e..d491327 100644
--- a/glib/glib/glib-init.c
+++ b/glib/glib/glib-init.c
@@ -342,12 +342,34 @@ glib_init (void)

 #if defined (G_OS_WIN32)

+HMODULE glib_dll;
+
+# ifdef G_HAS_CONSTRUCTORS
+
+#  ifdef G_DEFINE_CONSTRUCTOR_NEEDS_PRAGMA
+#   pragma G_DEFINE_CONSTRUCTOR_PRAGMA_ARGS(glib_init_ctor)
+#  endif
+G_DEFINE_CONSTRUCTOR(glib_init_ctor)
+
+static void
+glib_init_ctor (void)
+{
+  g_crash_handler_win32_init ();
+  g_clock_win32_init ();
+#ifdef THREADS_WIN32
+  g_thread_win32_init ();
+#endif
+  glib_init ();
+  /* must go after glib_init */
+  g_console_win32_init ();
+}
+
+# else
+
 BOOL WINAPI DllMain (HINSTANCE hinstDLL,
                      DWORD     fdwReason,
                      LPVOID    lpvReserved);

-HMODULE glib_dll;
-
 BOOL WINAPI
 DllMain (HINSTANCE hinstDLL,
          DWORD     fdwReason,
@@ -389,6 +411,8 @@ DllMain (HINSTANCE hinstDLL,
   return TRUE;
 }

+# endif
+
 #elif defined (G_HAS_CONSTRUCTORS)

 #ifdef G_DEFINE_CONSTRUCTOR_NEEDS_PRAGMA

The important bit is in glib-init.c. I don't think you need all the build flags, and I rudely disabled NLS to save some time. It probably needs a dtor too. But it basically works now.

[longnguyen2004]

There's a patch for glib 2.56.x that adds constructor support on Windows https://github.com/bincrafters/conan-glib/blob/fad3e50d0eaf69221c0ef65077823aee341cee25/patches/0002-win32-Prefer-the-use-of-constructors-over-DllMain.patch, not sure if it'll apply cleanly on later versions.

[hpjansson]

Oh great, that's basically the same thing, but done properly. Do you know if it's been proposed upstream?

[longnguyen2004]

After digging around glib's GitLab, I found this PR which has been merged a while ago, so theoretically it should work, if you use a new enough glib. Also, have you tried building glib separately, but produce static instead of shared libraries? Meson can be a bit painful with cross-compilation and such, but that should be the best option, instead of specifying the source files directly. I'll have a try at it myself.

[hpjansson]

I started doing cross-builds using MXE about a week ago and submitted this PR there. That wasn't too hard (biggest time investment was some Windows-specific code to query the console size and enable ANSI processing and Unicode).

Btw, this issue is sort of a continuation of #41 and is specifically about building with an embedded GLib. Sorry if that was unclear. Avoiding the external dep is something that gets brought up now and again, and if it can be done without too much of a maintenance burden, it would be nice to have as an option. That's a considerable "if", though.

[hpjansson]

Just checked, and MXE patches the GLib init as above. Makes you wonder how much time we'd have saved collectively if it'd been upstreamed back in 2013 :-)

[longnguyen2004]

At least it has been upstreamed, so we can finally have some sleep :)

[oakes]

Sweet thanks for this. I updated glib to 2.72.3 and it now works on windows (i had to remove a few functions and add an #include to get it to compile).

[longnguyen2004]

Did you try to build a static version of glib? Mine builds successfully with no errors

[oakes]

No I haven't tried that, I wanted to build everything with nim's build tool.

[hpjansson]

Long time no chat :-) Is there anything left to do here? I think our status on Windows is good now - I make static win32 native releases regularly.

The outstanding issues are:

• No development files (chafa.dll plus headers) since I don't have an MSVC env to test it in properly. I'd accept help with this, but probably best done in a new issue.
• No Cygwin support. Covered in #186.