search

servo / mozjs

Servo's SpiderMonkey fork
251 stars 118 forks source link

Building with MSYS2 and GCC #353

Open Uristqwerty opened 1 year ago

Uristqwerty commented 1 year ago

I decided to try building Servo with the MinGW I had installed. Not surprisingly, even with hacks and running some commands manually, it hasn't gone very well yet, but that isn't the focus of this issue. As mozjs is a dependency of Servo, and has its own troubles, I switched to focusing exclusively on it, and can report success! At the very least, the build command runs to conclusion with only warnings; I'm sharing my experiences now, before I've looked into whether it's possible to test whether the binary output actually runs without segfaulting or other, more insidious forms of error.

First, if you discard everything else, I believe this part's a legitimate discrepancy in the code: Both RuntimeFunctionCallback in mozjs/mozjs/js/src/jit/ProcessExecutableMemory.cpp and PRMJ_Now in mozjs/mozjs/js/src/vm/Time.cpp are declared static in their implementation but not in the function prototypes; GCC considers that an error.

Secondly, as clang is a requirement for bindgen, this is not purely GCC. Still, as far as I'm aware, GCC was chosen for both the host and target compiler environment variables, and so used throughout the process.

Everything else has to do with build scripts and dependencies. I believe this is the minimal setup; I ran through it in a fresh windows VM with no pre-installed developer tools. It may need a reboot or two, though I doubt anything more than exiting and re-starting the MinGW64 shell would truly be necessary, if even that. I made significant use of VM snapshots, though, choosing to fully shut the system down each time, so that I could revert build attempts, testing changes incrementally without and risk that lingering build detritus would affect the success or failure of the final, clean run.

Alright, on to the steps I took, starting outside of MSYS. I'll include both the general site, and specific download URLs for completeness, but I did little more than grab the then-latest.

https://www.msys2.org/

https://github.com/servo/mozjs

https://www.python.org/

The former's the whole point, the repo could as easily be fetched via git later, and the python is simply because one of the build scripts looks for c:/python27/, I'm neither aware of whether it was actually run at all, nor whether a 3.x release would have sufficed; I used MSYS to install python3 to satisfy other parts of the build process, so the 2.7 install might have gone entirely unused outside of that single check.

Within msys, a few more dependencies were required:

pacman -S mingw64/mingw-w64-x86_64-clang for bindgen. pacman -S git pacman -S make pacman -S mingw-w64-x86_64-python3 pacman -S m4 curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh Choosing not to install msvc-based prerequisites, selecting x86_64-pc-windows-gnu, and afterwards adding .cargo/bin/ to $PATH in ~/.bashrc.

Finally, it took a few hacks/patches to make the whole thing run. Someone more familiar with the tooling (perhaps even myself, given more days to learn its intricacies. Sharing this writeup so that others may benefit takes priority however) should be able to convert it into proper conditional checks, one way or another, but I certainly wasn't going to invest the effort before I even knew whether there was a disaster lurking later in the process.

The two functions mentioned earlier; I have yet to dig into the commit history to see whether the static keyword was deliberate, merely made the seemingly-most-plausible edit in the moment.

mozjs/mozjs/js/src/jit/ProcessExecutableMemory.cpp
 -PRUNTIME_FUNCTION RuntimeFunctionCallback(DWORD64 ControlPc, PVOID Context);
 +static PRUNTIME_FUNCTION RuntimeFunctionCallback(DWORD64 ControlPc, PVOID Context);

mozjs/mozjs/js/src/vm/Time.cpp
 -static int64_t PRMJ_Now() {
 +int64_t PRMJ_Now() {

mozjs/makefile.cargo still has a trace of MSYS compatibility left in it, but won't bother checking if WINDOWS is set. I'm not certain there are other target triples containing windows that this would be a problem for, but as a quick and obvious edit it worked as a preliminary change.

mozjs/makefile.cargo:
 -ifeq (windows,$(findstring windows,$(TARGET)))
 +ifeq (windows-msvc,$(findstring windows-msvc,$(TARGET)))

Most of the changes are in toolchain.configure, and are the most hackish, as it's less obvious how to check the target triple.

mozjs/mozjs/build/moz.configure/toolchain.configure
 -        #elif defined(__GNUC__) && !defined(__MINGW32__)
 +        #elif defined(__GNUC__)

 @def host_is_windows(host):
 +    return False

 @def is_windows(target, host):
 +    return False

 @         if host_or_target.kernel == "WINNT":
 -            supported = types = ("clang-cl", "clang")
 +            supported = types = ("clang-cl", "clang", "gcc")

 -set_config("PATH", altered_path)
 +# set_config("PATH", altered_path)

First is simple, there might be a good reason for not recognizing MinGW as GCC, or it might just be a leftover remnant of an old codepath, but letting it re-use the existing gcc logic is by far the most straightforward solution.

Next two, a quick hack to get on with the build rather than puzzle out how to make the functions examine the build triple in more detail, as they currently base their decisions on the target kernel being NT, which says nothing of the build tools chosen.

Without adding gcc to the list, it chose clang-cl as HOST_CC. I'm not sure what effect that would have ultimately had, but didn't want to risk mixing two different compilers at once.

Finally, a nasty one that I only figured out after first getting a successful build. I was seeing a duplicate C:\msys64\mingw64\bin\ appended to altered_path Well, almost duplicate, as it has a trailing backslash! This made it into target\debug\build\mozjs_sys-395560274efe6b08\out\build\config\autoconf.mk, where it got treated as a line continuation, changing the definition of PATH to end in C:\msys64\mingw64\bin PERL = C:/msys64/usr/bin/perl.exe. At that point, PATH was completely broken, the then-executing makefile couldn't even find make to invoke further sub-commands! At first, I manually edited the resulting autoconf.mk, which sufficed for the moment, but later went back and, after much grepping though the code, found the offending line. Ultimately, I haven't tested whether it was specifically the trailing backslash that broke the path entirely, or also that it got converted from MSYS's unix-style /c/foo:/usr/bar style to windows' drive letters and semicolons.

Not done with the patches and hacks, however.

mozjs/mozjs/config/baseconfig.mk
 -ifeq (a,$(firstword a$(subst /, ,$(abspath .))))
 -$(error MSYS make is not supported)
 -endif
mozjs/mozjs/js/src/old-configure
 -        if test -z "$CLANG_CC"; then
 -            { echo "configure: error: Firefox cannot be built with mingw-gcc and requires a mingw-clang toolchain to work" 1>&2; echo "configure: error: Firefox cannot be built with mingw-gcc and requires a mingw-clang toolchain to work" 1>&5; exit 1; }
 -        fi

 -        CXXFLAGS="$CXXFLAGS -Wno-incompatible-ms-struct"
 +        if test -n "$CLANG_CC"; then
 +            CXXFLAGS="$CXXFLAGS -Wno-incompatible-ms-struct"
 +        fi

A few of the scripts won't even try building under MSYS, easy enough to excise those conditionals. Skipping no-incompatible-ms-struct greatly cuts down on warnings, since GCC doesn't recognize the flag, though doesn't change whether the compiler invocations succeed or not.

mozjs/build.rs
 @    if target.contains("windows") {
 @        builder = builder.clang_arg("-fms-compatibility");
 @    }
 +
 +    if target.contains("windows-gnu") {
 +        builder = builder.clang_arg("-fgnuc-version=4.2.1");
 +        builder = builder.clang_arg("-fms-compatibility-version=19.00");
 +    }

Finally after everything else, I'm not sure why, but when bindgen invoked clang to process GCC's standard library headers, clang set neither __GNUC__ nor _MSC_VER, so vadefs.h failed to define va_list, causing a number of errors. Even forcing a GNUC version, it then did not understand char16_t in one of the C++ headers, though an internet search or two suggested that forcing a ms-comaptibility-version would work around that issue.

Altogether, this is enough to get it to build with MSYS2 and GCC; with any luck it builds correctly, and the crude hacks above aren't too difficult to polish into something that can be accepted into the repo. If nothing else, at least the next person to try can reference my own experiences to ease theirs.

Redfire75369 commented 1 year ago

According the to the Firefox Supported Build Targets, mingw-gcc is not supported due to clang-bindgen. Even mingw-clang is only tier 2.

If Firefox doesn't support it, I don't really think we should patch it so that it can. It would probably be a pain to maintain it.

Uristqwerty commented 1 year ago

I interpret that as "Well, you have to have clang installed anyway, so why bother?", though cannot track down further details. That line appears to have been there when the page was copied over from MDN, but (edit: the page as a whole) no longer exists there, so any change history that might reveal who made it, or reference a bug ID is obscured.

In the long run, that folks are actively working on Rust in the Linux kernel and gccrs suggests that one day bindgen might not require clang. Beyond that, many of the changes would be equally relevant to building with clang and MSYS2, as they relate more to the tooling outside of the specific compiler chosen.

Kreijstal commented 3 months ago

So why is this not patched in?

mrobinson commented 3 months ago

So why is this not patched in?

I think that @Redfire75369's comment is the important point. Adding support for a platform on top of upstream is a large maintenance burden and we're a small project. If it's the case that many of these changes are equally relevant for building with clang and MSYS2, a good place to submit them is in upstream SpiderMonkey. Then, as we update our version downstream we will get those features over time.