Building executables from source for Windows

[MihailCosmin]

Hi,

I see that you have on your website executables for windows (which I was using), but the instructions in INSTALL do not contain any info for Windows. How can I build the executables for windows? In order to get the latest changes to the brexcheck.

[kibook]

Hello @MihailCosmin,

Currently, I use MinGW on Windows to build the Windows executables. MinGW provides a minimal GNU environment for Windows, including standard POSIX utilities like Bourne shell. There are other, less minimal systems like Cygwin that would also work. With any of these installed, the instructions should be the same as on GNU/Linux.

I will add a section in the installation instructions with some basic guidelines about setting up MinGW as I have.

I can also start publishing pre-release builds. I create them fairly regularly for my own testing. I put up the build I did today from the latest commit here: https://github.com/kibook/s1kd-tools/releases/tag/3.1.18

[MihailCosmin]

Thanks for the updated windows executables.

I installed cygwin32 and installed coreutils, binutils, xxd, libxml2, libxslt and libexslt

Running make gives me a lot of errors, below some examples. There are many more. Am I missing something?

"cc -Wall -Werror -pedantic-errors -I ../common `pkg-config --cflags libxml-2.0 libxslt` -O3 -o s1kd-acronyms s1kd-acronyms.c ../common/s1kd_tools.c `pkg-config --libs libxml-2.0 libxslt`
In file included from /usr/i686-w64-mingw32/sys-root/mingw/include/_cygwin.h:14,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/_mingw.h:15,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/corecrt.h:10,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/crtdefs.h:10,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/stdio.h:9,
                 from s1kd-acronyms.c:1:
/usr/i686-w64-mingw32/sys-root/mingw/include/stddef.h:18:10: error: expected ';' before 'extern'
   18 |   _CRTIMP extern int *__cdecl _errno(void);
      |          ^~~~~~~
      |          ;
/usr/i686-w64-mingw32/sys-root/mingw/include/stddef.h:20:3: error: data definition has no type or storage class
   20 |   errno_t __cdecl _set_errno(int _Value);
      |   ^~~~~~~
/usr/i686-w64-mingw32/sys-root/mingw/include/stddef.h:20:3: error: type defaults to 'int' in declaration of 'errno_t' [-Wimplicit-int]
/usr/i686-w64-mingw32/sys-root/mingw/include/stddef.h:20:3: error: 'cdecl' attribute only applies to function types [-Werror=attributes]
/usr/i686-w64-mingw32/sys-root/mingw/include/stddef.h:20:19: error: expected ',' or ';' before '_set_errno'
   20 |   errno_t __cdecl _set_errno(int _Value);
      |                   ^~~~~~~~~~
/usr/i686-w64-mingw32/sys-root/mingw/include/stddef.h:21:3: error: data definition has no type or storage class
   21 |   errno_t __cdecl _get_errno(int *_Value);
      |   ^~~~~~~
/usr/i686-w64-mingw32/sys-root/mingw/include/stddef.h:21:3: error: type defaults to 'int' in declaration of 'errno_t' [-Wimplicit-int]
/usr/i686-w64-mingw32/sys-root/mingw/include/stddef.h:21:3: error: 'cdecl' attribute only applies to function types [-Werror=attributes]
/usr/i686-w64-mingw32/sys-root/mingw/include/stddef.h:21:19: error: expected ',' or ';' before '_get_errno'
   21 |   errno_t __cdecl _get_errno(int *_Value);
      |                   ^~~~~~~~~~
/usr/i686-w64-mingw32/sys-root/mingw/include/stddef.h:24:10: error: expected ';' before 'extern'
   24 |   _CRTIMP extern unsigned long __cdecl __threadid(void);
      |          ^~~~~~~
      |          ;
/usr/i686-w64-mingw32/sys-root/mingw/include/stddef.h:26:10: error: expected ';' before 'extern'
   26 |   _CRTIMP extern uintptr_t __cdecl __threadhandle(void);
      |          ^~~~~~~
      |          ;
/usr/i686-w64-mingw32/sys-root/mingw/include/stddef.h:26:18: error: type defaults to 'int' in declaration of 'uintptr_t' [-Wimplicit-int]
   26 |   _CRTIMP extern uintptr_t __cdecl __threadhandle(void);
      |                  ^~~~~~~~~
/usr/i686-w64-mingw32/sys-root/mingw/include/stddef.h:26:3: error: 'cdecl' attribute only applies to function types [-Werror=attributes]
   26 |   _CRTIMP extern uintptr_t __cdecl __threadhandle(void);
      |   ^~~~~~~
/usr/i686-w64-mingw32/sys-root/mingw/include/stddef.h:26:36: error: expected ',' or ';' before '__threadhandle'
   26 |   _CRTIMP extern uintptr_t __cdecl __threadhandle(void);
      |                                    ^~~~~~~~~~~~~~
In file included from /usr/i686-w64-mingw32/sys-root/mingw/include/crtdefs.h:10,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/stdio.h:9,
                 from s1kd-acronyms.c:1:
/usr/i686-w64-mingw32/sys-root/mingw/include/corecrt.h:118:13: error: 'errno_t' redeclared as different kind of symbol
  118 | typedef int errno_t;"

[kibook]

The errors seem to be in system headers and not the s1kd-tools code. Try compiling this minimal C program:

#include <stdio.h>

int main(int argc, char **argv)
{
    puts("Hello, world!");
    return 0;
}

Save it as "test.c" and then run:

$ cc test.c

If you get the same errors, there's something wrong with the environment.

[MihailCosmin]

No errors with this.

cc test.c created a.exe file Running a.exe in cmd outputs Hello, World!

[MihailCosmin]

I tried this

cc s1kd-brexcheck.c

And I got this:

s1kd-brexcheck.c:7:10: fatal error: libxml/tree.h: No such file or directory
    7 | #include <libxml/tree.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.

But libxml is installed, and tree.h is there. Where does the compiler look for include libraries? Do I need to set-up some other environment variables?

[kibook]

Try building with this:

$ make WARNING_FLAGS=

That will disable the very strict compiler options used by default (-Wall -Werror -pedantic-errors).

Maybe also try compiling that minimal C program again with this:

$  cc `pkg-config --cflags libxml-2.0 libxslt libexslt` test.c

If it fails then, maybe the compiler options being added for one of libxml2, libxslt or libexslt are causing the issue. Post the output of:

$ pkg-config --cflags libxml-2.0 libxslt libexslt

[MihailCosmin]

make WARNING_FLAGS= ends with many errors

ccpkg-config --cflags libxml-2.0 libxslt libexslttest.c also now has errors

output of pkg-config --cflags libxml-2.0 libxslt libexslt

-I/usr/i686-w64-mingw32/sys-root/mingw/include/libxml2 -I/usr/i686-w64-mingw32/sys-root/mingw/include

[kibook]

-I/usr/i686-w64-mingw32/sys-root/mingw/include

I think this may be the main culprit. I believe it is overriding the "normal" include path for standard headers like stdio.h with one that has some issues.

You could try seeing if it will compile without that include directive. The easiest way would probably be to tweak the Makefile for one of the tools. Go to the tools/s1kd-acronyms directory, and in the Makefile, change:

`pkg-config --cflags libxml-2.0 libxslt libexslt`

to:

-I/usr/i686-w64-mingw32/sys-root/mingw/include/libxml2

Then, in the tools/s1kd-acronyms directory, run make to build just that tool.

There's a good chance you'll just get different errors if there's something else in /usr/i686-w64-mingw32/sys-root/mingw/include that is required for libxml2, but it's worth a shot.

Because you're getting the errors with the minimal C program using libxml2, this is maybe more an issue with libxml2 on Cygwin than something specific to the s1kd-tools.

[MihailCosmin]

$ make
cc -Wall -Werror -pedantic-errors -I ../common -I/usr/i686-w64-mingw32/sys-root/mingw/include/libxml2 -O3 -o s1kd-acronyms s1kd-acronyms.c ../common/s1kd_tools.c -I/usr/i686-w64-mingw32/sys-root/mingw/include/libxml2

Now I didn't have the same errors. But something still doesn't work. I get a lot of messages like this:

undefined reference to `xsltParseStylesheetDoc'
undefined reference to `xsltApplyStylesheet'
undefined reference to `xmlFreeDoc'
undefined reference to `xsltFreeStylesheet'
..........

And ends with

collect2: error: ld returned 1 exit status
make: *** [Makefile:37: s1kd-acronyms] Error 1

[kibook]

Sorry, only replace the part that looks exactly like this in the Makefile:

`pkg-config --cflags libxml-2.0 libxslt libexslt`

The other call to pkg-config that looks like this:

`pkg-config --libs libxml-2.0 libxslt libexslt`

should be left alone.

The former provides options to the compiler (gcc), the latter is for the linker (ld).

[MihailCosmin]

Yup, that did it... exe file was generated.

[kibook]

Then it seems like /usr/i686-w64-mingw32/sys-root/mingw/include is not actually needed by libxml2, but is included in the pkg-config entry for libxml-2.0 in Cygwin for some reason.

There is a configuration file called libxml-2.0.pc somewhere in your environment that defines what arguments pkg-config provides for a package. You can find it with this command:

$ pkg-config --debug libxml-2.0 2>&1 | grep 'Parsing package file'

This should print a line with the location of the .pc file, like:

Parsing package file '/usr/lib/x86_64-linux-gnu/pkgconfig/libxml-2.0.pc'

Inside that file will be something like this (not exactly though since it depends on your environment):

prefix=/usr
exec_prefix=${prefix}
libdir=${prefix}/lib/x86_64-linux-gnu
includedir=${prefix}/include
modules=1

Name: libXML
Version: 2.9.1
Description: libXML library version2.
Requires:
Libs: -L${libdir} -lxml2
Libs.private:   -lz  -lm   -llzma
Cflags: -I${includedir}/libxml2

The Cflags: line determines what pkg-config outputs for the package when using the --cflags option.

If you remove the part that adds -I/usr/i686-w64-mingw32/sys-root/mingw/include, then it will just return the include path that works when using pkg-config --cflags libxml-2.0.

If it looks a bit different from my example and you're not sure what to change, post the contents of your libxml-2.0.pc file here and I can give you the "fixed" version.

By changing the configuration file for pkg-config, you won't have to manually change every tool's Makefile to workaround it, and afterwards you should be able to build everything with make in the top directory (🤞).

[MihailCosmin]

Some new info:

I installed cygwin in a clean Windows installation in Virtualbox. make started well (wtihout the changes as we did above) creating .exe files from s1kd-acronyms until s1kd-fmgen. Then I got an error:

cc -Wall -Werror -pedantic-errors -I ../common `pkg-config --cflags libxml-2.0` -O3 -o s1kd-icncatalog s1kd-icncatalog.c ../common/s1kd_tools.c `pkg-config --libs libxml-2.0` -lregex2
/usr/lib/gcc/i686-pc-cygwin/9.3.0/../../../../i686-pc-cygwin/bin/ld: cannot find -lregex2
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:26: s1kd-icncatalog] Error 1

I cannot find any info on this "lregex2"

[kibook]

-lregex2 means basically "link with the regex2 library". regex2 is the GNU implementation of the POSIX regular expression functions. On some systems it may not be necessary to link this, but it was needed for MinGW. I'm not sure about Cygwin though.

First, try commenting out this part in the Makefile for s1kd-icncatalog:

ifeq ($(OS),Windows_NT)
        LDFLAGS+=-lregex2
endif

(to comment out, place # before each line)

As you can see, I made the condition for linking it whether you're on Windows or not. If Cygwin doesn't need it, I will look into making this more specific to a MinGW environment instead of just any Windows environment.

If commenting out/removing that part causes the build of s1kd-icncatalog to fail with errors like:

...: undefined reference to `regcomp'
...: undefined reference to `regexec'
...: undefined reference to `regfree'

Then you do need to link regex2, and you'll have to get the Windows binary for it, regex2.dll.

I don't know if there's a way to get it via package manager in Cygwin, but you can download the Windows binaries (.dll) directly here: http://gnuwin32.sourceforge.net/packages/regex.htm. It's also bundled in the binary packages for the s1kd-tools.

Place regex2.dll somewhere ld looks for libraries. You can find the default paths with this:

$ ld --verbose | grep SEARCH_DIR

Typically it will be directories like /usr/lib, /usr/local/lib, etc. Place regex2.dll in one of those.

[MihailCosmin]

Commenting those lines worked (on the clean virtualbox windows installation) Global make continued until s1kd-ref

There I am getting more errors:

cc -Wall -Werror -pedantic-errors -I ../common `pkg-config --cflags libxml-2.0 libxslt` -O3 -o s1kd-ref s1kd-ref.c ../common/s1kd_tools.c `pkg-config --libs libxml-2.0 libxslt`
In file included from s1kd-ref.c:6:
s1kd-ref.c: In function ‘new_smc_ref’:
s1kd-ref.c:291:29: error: array subscript has type ‘char’ [-Werror=char-subscripts]
  291 |    } else if (s && isdigit(s[1])) {
      |                            ~^~~
s1kd-ref.c: In function ‘new_pm_ref’:
s1kd-ref.c:485:29: error: array subscript has type ‘char’ [-Werror=char-subscripts]
  485 |    } else if (s && isdigit(s[1])) {
      |                            ~^~~
s1kd-ref.c: In function ‘new_dm_ref’:
s1kd-ref.c:734:29: error: array subscript has type ‘char’ [-Werror=char-subscripts]
  734 |    } else if (s && isdigit(s[1])) {
      |                            ~^~~
s1kd-ref.c: In function ‘new_dml_ref’:
s1kd-ref.c:985:29: error: array subscript has type ‘char’ [-Werror=char-subscripts]
  985 |    } else if (s && isdigit(s[1])) {
      |                            ~^~~
s1kd-ref.c: In function ‘trim’:
s1kd-ref.c:1419:17: error: array subscript has type ‘char’ [-Werror=char-subscripts]
 1419 |  while (isspace(*str)) str++;
      |                 ^~~~
s1kd-ref.c:1424:30: error: array subscript has type ‘char’ [-Werror=char-subscripts]
 1424 |  while (end > str && isspace(*end)) end--;
      |                              ^~~~
s1kd-ref.c: In function ‘transform_ref’:
s1kd-ref.c:16:19: error: field precision specifier ‘.*’ expects argument of type ‘int’, but argument 5 has type ‘regoff_t’ {aka ‘long int’} [-Werror=format=]
   16 | #define PROG_NAME "s1kd-ref"
      |                   ^~~~~~~~~~
s1kd-ref.c:21:20: note: in expansion of macro ‘PROG_NAME’
   21 | #define INF_PREFIX PROG_NAME ": INFO: "
      |                    ^~~~~~~~~
s1kd-ref.c:1510:19: note: in expansion of macro ‘INF_PREFIX’
 1510 |   fprintf(stderr, INF_PREFIX "%s: Found %s ref %.*s\n", path, type, eo - so, (*content) + so);
      |                   ^~~~~~~~~~
s1kd-ref.c:1510:50: note: format string is defined here
 1510 |   fprintf(stderr, INF_PREFIX "%s: Found %s ref %.*s\n", path, type, eo - so, (*content) + so);
      |                                                ~~^~
      |                                                  |
      |                                                  int
cc1: all warnings being treated as errors
make[1]: *** [Makefile:26: s1kd-ref] Error 1

[kibook]

Commenting those lines worked

Great! So it seems the regex functions are already linked in Cygwin. The same was true of all the Linux systems I've compiled on, just not MinGW.

Can you post the output of this command:

$ echo $OSTYPE

I believe for Cygwin it should be 'cygwin', while for MinGW it is 'msys', so I can change that part of the Makefile to:

ifeq ($(OSTYPE),msys)
    LDFLAGS+=-lregex2
endif

There I am getting more errors:

These are indeed errors with the source of s1kd-ref (and also s1kd-refs). Technically, they are warnings about potential issues with the way isspace/isdigit are used and not errors, but by default the compiler options abort on warnings as well as errors.

These issues should be fixed in e533e93.

[MihailCosmin]

The output of $OSTYPE is as you said cygwin.

$ echo $OSTYPE
cygwin

[kibook]

Thanks! I changed the check in the Makefiles of the relevant tools (149be43).

[MihailCosmin]

Hi,

Downloaded the latest release 3.2.0 and make worked until s1kd-ref. Then a few errors:

cc -Wall -Werror -pedantic-errors -I ../common `pkg-config --cflags libxml-2.0 libxslt` -O3 -o s1kd-ref s1kd-ref.c ../common/s1kd_tools.c `pkg-config --libs libxml-2.0 libxslt`
s1kd-ref.c: In function ‘transform_ref’:
s1kd-ref.c:16:19: error: field precision specifier ‘.*’ expects argument of type ‘int’, but argument 5 has type ‘long int’ [-Werror=format=]
   16 | #define PROG_NAME "s1kd-ref"
      |                   ^~~~~~~~~~
s1kd-ref.c:21:20: note: in expansion of macro ‘PROG_NAME’
   21 | #define INF_PREFIX PROG_NAME ": INFO: "
      |                    ^~~~~~~~~
s1kd-ref.c:1510:19: note: in expansion of macro ‘INF_PREFIX’
 1510 |   fprintf(stderr, INF_PREFIX "%s: Found %s ref %.*s\n", path, type, (int) eo - so, (*content) + so);
      |                   ^~~~~~~~~~
s1kd-ref.c:1510:50: note: format string is defined here
 1510 |   fprintf(stderr, INF_PREFIX "%s: Found %s ref %.*s\n", path, type, (int) eo - so, (*content) + so);
      |                                                ~~^~
      |                                                  |
      |                                                  int
cc1: all warnings being treated as errors
make[1]: *** [Makefile:26: s1kd-ref] Error 1
make[1]: Leaving directory '/cygdrive/c/Users/Cosmin/Downloads/s1kd-tools-master 3.2.0/s1kd-tools-master/tools/s1kd-ref'
make: *** [Makefile:6: tools/s1kd-ref] Error 2

[kibook]

Should be fixed by 300e857. I missed that (int) eo - so only casts eo. Corrected to (int) (eo - so).

[MihailCosmin]

Hi,

Downloaded the source for 3.2.1. and make finished creating all executables. But the .dlls are not found.

When trying to use any of the .exe it tells me some dlls are missing. But not the ones you have in your windows release. The .dlls I needed were:

cygexslt-0.dll
cyggcrypt-20.dll
cygiconv-2.dll
cyglzma-5.dll
cygwin1.dll
cygxml2-2.dll
cygxslt-1.dll
cygz.dll

I found these in the cygwin folder and copied them. Is there another way of getting the needed .dlls during building?

[kibook]

Which DLLs are needed to create a "standalone" distributable package depends on the environment you use to build and the particular versions of libraries used. So, it's normal that your build from Cygwin will require different DLLs. You shouldn't need to copy any DLLs if you are running the tools from within the Cygwin environment, though, since their locations should be in the search path for shared libraries. Copying the DLLs is only necessary if you want to run the tools outside of Cygwin or on a "normal" Windows install.

For the s1kd-tools_*_win32.zip package, I essentially just did what you have, tested them on a "normal" Windows install, outside of the MinGW environment, to determine which DLLs I had to bundle.

There are tools to get a list of shared libraries used by an executable, like ldd or objdump:

$ objdump -x tools/s1kd-*/s1kd-*.exe | grep 'DLL Name' | sort -u
        DLL Name: KERNEL32.dll
        DLL Name: libexslt-0.dll
        DLL Name: libxml2-2.dll
        DLL Name: libxslt-1.dll
        DLL Name: msvcrt.dll
        DLL Name: regex2.dll

Cygwin in particular seems to have one called cygcheck, which can be used like:

$ cygcheck program.exe

DLLs may also link to other DLLs, so you need to check them for dependencies as well. Doing a quick search, I found a tool called Dependencies that can recursively search for all DLLs a program depends on. You don't necessarily need all the DLLs it will list, though, since some are part of the "normal" Windows install (like KERNEL32.dll).