skeeto / w64devkit

Portable C and C++ Development Kit for x64 (and x86) Windows
The Unlicense
2.7k stars 185 forks source link

Windows XP Support ? #11

Closed a-p-jo closed 2 years ago

a-p-jo commented 2 years ago

Windows XP is the earliest target that most devs care that their executable build for. It's still very widely used and simply abandoning anything older than Vista/7 is not a great option.

It's also an incredibly easy to run in a VM for a *nix user to test/compile for Windows platforms. Not to mention nearly anything that runs on XP will run on the latest Windows just fine. So it would be incredibly useful if w64devkit binaries ran on XP.

So, would it be possible to release XP-compatible binaries ? Or are there major impediments in achieving this ...

Running on a XP Pro SP3 Version 2002 VM gives :

image

skeeto commented 2 years ago

The error is because w64devkit is strictly 64-bit: All binaries are x86-64 and the toolchain itself only produces x86-64 binaries. It probably does work on Windows XP x64 Edition, though I don't have a copy available to check.

A maximum compatibility approach would be i686 w64devkit binaries and to enable multilib support in the toolchain so that it can produce both i686 and x86-64 binaries. I think it would also need two different GDB builds, one for each architecture. The downsides to the toolchain itself being i686 would be limited ability to link large C++ programs (LLVM infamously cannot be linked on 32-bit architectures), and it would be a little slower (the x86-64 ISA and x64 ABI are simply more efficient), but that's probably it.

Some BusyBox shell features don't work prior to Windows 10 since they depend on Windows 10's console improvements — finally coming to parity with the rest of the world.

The major hurdle is that the community has already moved on. This project is very much a built-on-the-shoulders-of-giants thing, and I depend on the broader open source community to solve issues, leaving only a handful to fix myself. Autoconf configure scripts operate on a combination of luck and duct tape. They tend to fail as soon as they face anything off the community's beaten path, and an i686 Windows cross compiler is no longer on that path. Out of curiosity, I tried to build a pure i686 w64devkit, but I'm running into bugs in GCC's configure scripts. If you wanted to try it yourself:

The first snag is the libgcc configure script running the compiler with the wrong options. I expect that more and more snags will appear in future releases of the included software since nobody's testing this path.

As for my personal philosophy: I just have no interest in 32-bit Windows, and aside from running old games, I consider it dead and buried. This is convenient for w64devkit since I can exclude 32-bit support, making for a slimmer, simpler, focused toolchain.

Despite the effort I've put into this project, I actually hate Windows. It's an ugly, crummy, miserable platform. w64devkit is me carving out a little piece of sanity. Professionally there's no escaping Windows, and it's still what most people are using in the desktop / laptop domain, so there are significant benefits in taming its worst tendencies.

Since Microsoft dropped Windows 7 support, I now get to ignore everything before Windows 10. Professionally I no longer deal with anything older, and the majority has moved on, along with all the benefits. Windows 10 may be bloated, slow, mismanaged, and increasingly loaded with spyware, dark patterns, propaganda, and advertising — about which Microsoft ought to be deeply embarrassed — but at least the console is no longer total garbage. What was once a major, time-sucking deficiency is now just a nuisance.

A great future for w64devkit would be irrelevancy: Either the Windows platform is dead, or it finally became tolerable on its own. I'm not holding my breath on either.

skeeto commented 2 years ago

Curiosity got the best of me, and indeed the latest w64devkit works just fine (aside from the expected console limitations) on Windows XP x64 Edition. w64devkit-xp-x64

skeeto commented 2 years ago

Impressed with how well it was working, I spent some time actually programming on Windows XP with w64devkit over the past couple of days. The main downside was missing quality-of-life features that I now take for granted, like the text box on the start menu.

However, the most striking difference was how much faster and snappier Windows XP feels compared to Windows 10. The difference is night and day. I knew the performance situation had gotten bad over the years, but seeing and using them side by side really put it in perspective. The console and Vim come up so quickly on XP that it almost feels like something's wrong, but that's just how much I'm accustomed to bad performance in modern software. I've captured a comparison recording below. These are both virtual machines with the same configuration running on the same machine recorded one after another.

https://user-images.githubusercontent.com/29988/133713736-355215ba-c679-42f9-b36f-d8f61ba3e23e.mp4

Walking through this frame-by-frame:

Now shown: Bringing up the explorer window with W-e is similarly sooo much better on Windows XP, essentially instant. While on Windows 10 it takes nearly a second to come up.

a-p-jo commented 2 years ago

It's always nice to get an involved response to an issue , thanks :)

The major hurdle is that the community has already moved on. This project is very much a built-on-the-shoulders-of-giants thing, and I depend on the broader open source community to solve issues, leaving only a handful to fix myself. Autoconf configure scripts operate on a combination of luck and duct tape. They tend to fail as soon as they face anything off the community's beaten path, and an i686 Windows cross compiler is no longer on that path

Do note that Code::Blocks still supports 32-bit XP binaries as well as produces them from I can understand.

But I understand. TBH, Being able to compile to 32-bit XP compatible binaries from 64-bit w64devkit would still be quite aweomse, because I want to say goodbye to the bloated, fragile systems of both Cygwin and Visual Studio.

AFAIK, using :

#define _WIN32_WINNT _WIN32_WINNT_WINXP

works cleanly with w64devkit's gcc on my Windows 7 machine , but from what you've said above I suspect the binaries are 64-bit.

Despite the effort I've put into this project, I actually hate Windows

I understand and agree. Unfortunately, that 80% market share isn't going to go away overnight and while I don't distribute binaries (at all) I'd like it if I could test and make sure my programs compiled and ran on Windows should someone want that. Personally, I just use *nix because all I need to do is program C and Python in neovim (any plans to offer neovim ;) ?)

Curiosity got the best of me, and indeed the latest w64devkit works just fine (aside from the expected console limitations) on Windows XP x64 Edition.

This is perfect. I'll switch to 64-bit XP for my VM.

the most striking difference was how much faster and snappier Windows XP feels compared to Windows 10. The difference is night and day.

Indeed, XP was the last trivially embeddable, virtualizable, reliable OS that M$ produced. Windows Vista/7 are also exponentially more resource greedy. The only sensible option for a cheap Windows VM is XP. This is why I'm glad that w64devkit will run on it, at least the 64-bit version.

You realise how cool this is ?

Visual Studio 2010 is the last msvc compiler I can get there, and that doesn't properly support ISO C89 . Code::Block's latest has gcc 8.1, and is way larger than what I need , and has no CLI interface that I could get to work. Cygwin is clunky.

w64devkit is brilliantly simple and just what I need. Thanks again for it, as well as you awesome blog, nullprogram.

a-p-jo commented 2 years ago

If I might also ask, where did you get an installation copy for XP ? Those are not so easily available, especially the 64-bit one

skeeto commented 2 years ago

Being able to compile to 32-bit XP compatible binaries from 64-bit w64devkit would still be quite aweomse

As I mentioned, a core goal is being slim and focused. Multilib support goes against both: There would need to be two copies of all runtime libraries, 32-bit and 64-bit, plus the extra toolchain machinery to use it all. However, 32-bit Windows development is now niche. Most people don't need or want this feature, so I don't want to make them pay for it.

(I really do take size very seriously. Examine older releases and you'll see that, opposite the usual software trend, w64devkit has actually gotten smaller over time despite gaining features. The next release will be the smallest yet.)

However, especially after my experience here, I am not opposed to figuring a pure 32-bit w64devkit build: all 32-bit tools, producing only 32-bit binaries. I had listed before the required Dockerfile changes, but there are still issues to be solved. If it works well I might even build an official 32-bit release.

define _WIN32_WINNT _WIN32_WINNT_WINXP

Thanks for the reminder. If there was working 32-bit support, official or unofficial, this is surely worth documenting. I built Lua 5.4 on XP, but w64devkit linked against a newer msvcrt.dll, and so it wouldn't run on XP due to a missing symbol. I didn't try, but I believe this would have fixed it.

where did you get an installation copy for XP ?

This was literally the first result when I went looking a few days ago:

https://archive.org/details/windowsxpprox64english

Note: I have no idea if these ISOs are clean and trustworthy. I operated my VM assuming it was running malware, trusted nothing it produced, and completely destroyed it when I was done. That's good enough for testing, but not for real use. The archive is nearly two years old, so if there is malware then at least its two years old.

a-p-jo commented 2 years ago

It's always refreshing to see fellow developers who care about binary sizes being reasonable these days.

However, especially after my experience here, I am not opposed to figuring a pure 32-bit w64devkit build: all 32-bit tools, producing only 32-bit binaries

Interesting, and very nice if it could be achieved. Here's hoping those autoconf srcipts can be arm-twisted to work. On the other hand, your Dockerfile seems to be cross-compiling the toolchain on Debian. How about natively building it on Windows , would that help the fragile build configuration scripts behave better ? Perhaps with a cygwin installation to compiler it to bootstrap then use the old devkit to compile the new releases (something tells me the cygwin people ought to be able to compile their binaries on their setup with all that bloat they have).

While you do not use Windows regularly and detest it in general, the above could be tried in a VM, depending on your hardware.

Then again , I don't know much about build systems or building large (GNU-sized ;) ) projects and am only guessing.

w64devkit linked against a newer msvcrt.dll, and so it wouldn't run on XP due to a missing symbol. I didn't try, but I believe this would have fixed it.

At the very least it should cause the build to fail if the source requires dependencies that are not possible to furnish on Windows XP.

And another Windows-XP-support question if you don't mind - How did you install anything onto the VM ? XP comes with an outdatedIE that can't load basic modern websites anymore.

skeeto commented 2 years ago

How about natively building it on Windows

Building GCC requires a full-blown, modern unix-like environment — and not just a thin veneer like w64devkit — so this would absolutely require either Cygwin or (better these days) MSYS2. I really don't want to get into that, and w64devkit exists specifically for me to avoid these environments. My official releases are all built on Linux and never actually make contact with Windows until someone downloads them there.

https://gcc.gnu.org/wiki/WindowsBuilding

Unfortunately this also means w64devkit will never be self-hosting.

(You can build w64devkit on Windows via Docker Desktop, though of course this doesn't help with XP.)

How did you install anything onto the VM ?

Two different approaches:

skeeto commented 2 years ago

Some good news: After going through the Mingw-w64 configuration documentation to see if I missed anything, I found --disable-lib64, and suddenly I can successfully build a pure, working 32-bit w64devkit. It's actually a about 5% smaller than the 64-bit version, which isn't too surprising.

My first attempt didn't run on Windows XP. I ran into a small hiccup with a Mingw-w64 regression:

https://sourceforge.net/p/mingw-w64/bugs/821/

I ported the old fix forward, and now it works on i686 Windows XP. My "i686" branch has the necessary changes, and I'm tweaking the normal Dockerfile so that hopefully the i686 version does not need so many changes.

raysan5 commented 2 years ago

@skeeto Wow! That's very nice! So, #6 could be addresses with this improvement?

skeeto commented 2 years ago

Update: I've made tweaks to the Dockerfile to simplify this "i686" variant as I'm calling it. I've added patches to src/ for each variant which tweak the Dockerfile to add/remove/change features. Multiple variant patches can be applied at once (more below), so you could, say, also have a 32-bit Fortran compiler.

I've also built an official 32-bit 1.9.0 release with Windows XP support, backporting the necessary changes to the 1.9.0 tag, which you can find in the Releases. Tell me how that works for you.

Yup, @raysan5, I think this is what you were looking for earlier this year. Currently on master with the "mini" and "i686" patches applied, and optimized with advancecomp (advzip -z4), the .zip is just 50MB and runs on everything at least as far back as Windows XP.

a-p-jo commented 2 years ago

now it works on i686 Windows XP ... I've also built an official 32-bit 1.9.0 release with Windows XP support

Incredible ! Thanks a lot !

Tell me how that works for you.

Sure will :)

EDITS :

raysan5 commented 2 years ago

@skeeto Wow! This is really amazing! Thank you very much for working on it! I'll be using your release for the next raylib Windows Installer package! Thanks! :D

a-p-jo commented 2 years ago

@skeeto Would it be possible to include gfortran in the i686 build ?

raysan5 commented 2 years ago

@a-p-jo Personally, I prefer a minimal package with only the minimal required tools.

skeeto commented 2 years ago

I've uploaded a "fortran-i686" build for 1.9.0.

Note: After some testing I've found that gfortran-compiled programs have absolutely abysmal performance. I'm not sure it's just emitting bad x87 code, or if x87 is really just that bad. (I didn't come onto the scene until x87 was obsolete.) This isn't particular to w64devkit: my Debian system's i686-linux-gnu-gfortran is just as bad. I needed to compile with "-msse2" in order to get certain programs to run to completion within a reasonable time.

a-p-jo commented 2 years ago

I prefer a minimal package with only the minimal required tools

Understandable, @raysan5 . Skeeto's idea of separate builds comes handy here.

I've found that gfortran-compiled programs have absolutely abysmal performance

That's strange.

The C/C++ programs are reasonably fast though, yes ? Don't gcc/g++/gfortran have the same backend...

P.S: Now that there is a 32-bit build, I suppose you can close this issue anytime you wish to.

skeeto commented 2 years ago

If you wanted to try it for yourself, here's a simple example:

https://github.com/skeeto/scratch/blob/master/misc/mandelbrot.f90

On x86-64 this takes about a second, and even on my Raspberry Pi 4, both aarch64 and arm (gnueabihf), it takes just a few seconds. On i686 (any OS) without SSE2 it takes my x86-64 laptop around 7 minutes. With -msse2 it's back down to a second.

I have not yet found any C programs with such dramatic difference in performance on i686, which makes me suspect gfortran is somehow just bad at x87 in particular. However, in C there still is a noticeable difference in float-intensive programs between x87 and SSE2. For instance:

https://github.com/skeeto/scratch/blob/master/animation/galton.c

On my x86-64 desktop, compiled with my i686 w64devkit, default arch (no SSE2), this stutters since it can't quite keep up with 60 FPS. Enable SSE2, which I've been doing with -march=pentium4 for a more practical baseline, and it does fine. Small difference, but noticeable.

As such, my current plan is to target "pentium4" for i686 builds, both for the release itself and as the default target arch, so that SSE2 is enabled by default.

https://github.com/skeeto/w64devkit/blob/master/src/variant-i686.patch

a-p-jo commented 2 years ago

my current plan is to target "pentium4" for i686 builds

Considering the first Windows XP came out near the terminal end of 2001, and the Pentium 4 series shipped at the end of 2000, and they were both dropped in mid 2008, that seems like a sensible decision, great work !

This seems to explain the floating point performance difference with and without SSE2 in the C code.

Considering the mandelbrot code in Fortran uses a lot of real and complex types, maybe there is some language strictness or lack of developer interest so maybe gfortran does not allow some performance trickery that gcc is able to employ, making the hardware level performance difference glaringly visible.