rmyorston / busybox-w32

WIN32 native port of BusyBox.
https://frippery.org/busybox
Other
674 stars 124 forks source link

`uname` reports incorrect os release/version #366

Closed naksyl closed 8 months ago

naksyl commented 1 year ago

On my Windows 11 machine uname -a reports:

~ $ uname -a
Windows_NT lenovo 6.2 9200 x86_64 MS/Windows

which suggest Windows 8 OS.

After some digging I found on Microsoft docs:

With the release of Windows 8.1, the behavior of the GetVersionEx API has changed in the value it will return for the operating system version. The value returned by the GetVersionEx function now depends on how the application is manifested.

Applications not manifested for Windows 8.1 or Windows 10 will return the Windows 8 OS version value (6.2). Once an application is manifested for a given operating system version, GetVersionEx will always return the version that the application is manifested for in future releases.

Small demo

uname.c

#include <Windows.h>
#include <stdio.h>

int main()
{
    OSVERSIONINFO os_info;

    memset(&os_info, 0, sizeof(OSVERSIONINFO));
    os_info.dwOSVersionInfoSize = sizeof(OSVERSIONINFO);

    GetVersionEx(&os_info);
    printf("release: %u.%u\n", (unsigned int)os_info.dwMajorVersion,
            (unsigned int)os_info.dwMinorVersion);
    printf("version: %u\n", (unsigned int)os_info.dwBuildNumber);

    return 0;
}

uname.manifest

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
  <description>uname demo</description>
  <assemblyIdentity version="1.0.0.0" name="uname"/>
      <compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1">
        <application>
            <!-- Windows 10 and Windows 11 -->
            <supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}"/>
            <!-- Windows 8.1 -->
            <supportedOS Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}"/>
            <!-- Windows 8 -->
            <supportedOS Id="{4a2f28e3-53b9-4441-ba9c-d69d4a4a6e38}"/>
            <!-- Windows 7 -->
            <supportedOS Id="{35138b9a-5d96-4fbd-8e2d-a2440225f93a}"/>
            <!-- Windows Vista -->
            <supportedOS Id="{e2011457-1546-43c5-a5fe-008deee3d3f0}"/> 
        </application>
    </compatibility>
</assembly>

uname.rc

1 24 "uname.manifest"

Test

~/ $ windres uname.rc uname.o
~/ $ gcc uname.c uname.o -o uname
~/ $ ./uname
release: 10.0
version: 22621

This still does not report Windows 11 but at least version number is correct. I have no older Windows installed right now but running this uname in compatibility mode print compatibility OS as expected.

References:

rmyorston commented 1 year ago

Indeed, the binary needs to include the app manifest to report the correct OS version.

Cygwin has a windows-default-manifest package containing the required manifest. This has also been adopted by MSYS2 and Fedora. I use the latter to build the binaries I release, so they include the manifest.

I suppose it might be useful to include a copy of the manifest in the busybox-w32 source for toolchains that don't have such a default package.

naksyl commented 1 year ago

I use the latter to build the binaries I release, so they include the manifest.

My bad. Just testing wrong binary and won the jackpot

~ $ for b in busybox*; do echo -n $b" ":; ./$b uname -a; done
busybox-w64-FRP-3812-g12e14ebba.exe :Windows_NT lenovo 10.0 22621 x86_64 MS/Windows
busybox-w64-FRP-3902-g61e53aa93.exe :Windows_NT lenovo 10.0 22621 x86_64 MS/Windows
busybox-w64-FRP-4264-gc79f13025.exe :Windows_NT lenovo 10.0 22621 x86_64 MS/Windows
busybox-w64-FRP-4487-gd239d2d52.exe :Windows_NT lenovo 10.0 22621 x86_64 MS/Windows
busybox-w64-FRP-4621-gf3c5e8bc3.exe :Windows_NT lenovo 10.0 22621 x86_64 MS/Windows
busybox-w64-FRP-4716-g31467ddfc.exe :Windows_NT lenovo 10.0 22621 x86_64 MS/Windows
busybox-w64-FRP-4784-g5507c8744.exe :Windows_NT lenovo 10.0 22621 x86_64 MS/Windows
busybox-w64-FRP-4881-ga6c5fd4eb.exe :Windows_NT lenovo 10.0 22621 x86_64 MS/Windows
busybox-w64-FRP-4882-g6e0a6b7e5.exe :Windows_NT lenovo 10.0 22621 x86_64 MS/Windows
busybox-w64-FRP-5007-g82accfc19.exe :Windows_NT lenovo 10.0 22621 x86_64 MS/Windows
busybox-w64-PRE-5195-g2af141a2c.exe :Windows_NT lenovo 10.0 22621 x86_64 MS/Windows
busybox-w64u-PRE-5195-g2af141a2c.exe :Windows_NT lenovo 6.2 9200 x86_64 MS/Windows
rmyorston commented 1 year ago

Hold on. The busybox-w64u binary should have the manifest too.

It looks as though the UTF-8 manifest and the default app manifest don't want to co-exist.

avih commented 1 year ago

Cygwin has a windows-default-manifest package containing the required manifest. This has also been adopted by MSYS2 and Fedora. I use the latter to build the binaries I release, so they include the manifest.

You mean the manifest is part of the toolchain? I don't think it exists or gets used when building with w64devkit.

I'm getting on win10: Windows_NT alap2 6.2 9200 x86_64 MS/Windows.

I suppose it might be useful to include a copy of the manifest in the busybox-w32 source for toolchains that don't have such a default package.

+1

Does it have any effect on running windows xp?

naksyl commented 1 year ago

It looks as though the UTF-8 manifest and the default app manifest don't want to co-exist.

Maybe it should be in a single file - UTF8 overwrites deafult?

rmyorston commented 1 year ago

You mean the manifest is part of the toolchain?

Some toolchains provide it. w64devkit doesn't.

Does it have any effect on running windows xp?

No, only the UTF-8 section of the manifest upsets Windows XP.

Maybe it should be in a single file

Yes, it looks as though everything needs to be in one manifest. I'm putting together a configuration that should cover all cases.

avih commented 1 year ago

You mean the manifest is part of the toolchain?

Some toolchains provide it. w64devkit doesn't.

Does it have any effect on running windows xp?

No, only the UTF-8 section of the manifest upsets Windows XP.

Maybe it should be in a single file

Yes, it looks as though everything needs to be in one manifest. I'm putting together a configuration that should cover all cases.

You mean additional configurations?

Shouldn't it be included in all configs by default, while UTF8_MANIFEST would usea different manifest file which merges the utf8 and the win versions ones?

naksyl commented 1 year ago

Not sure that busybox support it, but there are other usefull manifests:

rmyorston commented 1 year ago

OK, we have new prerelease binaries.

avih commented 1 year ago

Hmm, so a non-unicode 32/64 build with w64devkit or any other setup without a toolchain manifest would, by default, result in sub-par win-versions behavior, while a 64u build will have the better win-versions behavior by default?

FWIW, busybox.exe which ships with w64devkit (and I believe is built in a debian docker image) also doesn't have such manifest that I can tell (and also reports 6.2 on win10).

Why not make the app-manifest default also for the non-utf8 builds?

rmyorston commented 1 year ago

Why not make the app-manifest default also for the non-utf8 builds?

Because it increases the size of the binary by 1.5KB if the toolchain also has a manifest.

To get the best behaviour and the best size in any particular case might require manual intervention.

avih commented 1 year ago

Because it increases the size of the binary by 1.5KB if the toolchain also has a manifest.

Huh, so, does it include both the toolchain manifest and the app manifest?

Otherwise, I'd expect that if only one manifest is included then the result size would be the same regardless of the source of the manifest (and obviously assuming the manifests have the same size, which I presume is the case).

You can use perc -L busybox.exe to check which resources and manifests a binary has.

avih commented 1 year ago

Hmm.. I can confirm the 1.5K bigger with app-manifest in msys2 64 compared to (existing) toolchain-manifest.

It's weird though, the app-manifest is actually 2 bytes smaller than the toolchain manifest (1165/1167), and yet the binary is 1.5K bigger.

Both of them have a single manifest resource with ID 1, though the default one has a neutral language (0), while the app/utf8 manifest is en-us (1033). I wonder whether language 0 of the app/utf8 manifests would override the default manifest better.

I compared the binaries, and the app-manifest one has some additional block of zeroes.

This https://github.com/msys2/MSYS2-packages/issues/454 though from 2016, and this apparently related gcc bug report suggest that this is a gcc feature, and that using a custom manifest together with the default toolchain manifest can have bad consequences, like corupted executable (badly constructed sections). I'm guessing this has been resolved since, but there are still reported issues as recent as few months ago at the end of that msys2 link.

I'm guessing that the fix didn't actually disable/merge the default manifest if an external one exists, but rather ensured the binary ends up valid (upx can compress it without complaining, and strip doesn't reduce its size further - both happened when the issue was reported initially).

I wonder if there's a way to disable the toolchain manifest...

EDIT: Huh... copying the manifest from busybox to itself with perc, like so (the manifest is the 10th resource in perc -L ...), which should be no-op:

perc -x 10 ./busybox.exe | perc -a - 10 ./busybox.exe

Makes it small again.

So this makes me think that gcc still doesn't construct the resources block good enough. perc uses the windows API to access and update the resources, so I'm guessing that using this API "fixes" the resources construction at the binary, and the file becomes the size it should have been.

The same thing (the binary becomes a bit smaller) happens when doing the same supposedly-no-op copy of the manifest to itself with the pre-release 64u binary. We really need to find a way to either disable the default manifest or be able to override it completely.

avih commented 12 months ago

copying the manifest ...

Apparently it's enough to do begin+end UpdateResource to "fix" the binary and make it smaller after using a custom manifest when the toolchain has a default manifest, like with this program:

// touch-resources.c
#include <windows.h>
#include <stdio.h>

int main(int argc, char **argv) {
    HANDLE h;

    if (argc != 2)
        fprintf(stderr,"Usage: %s FILE   Do Begin+End UpdateResource\n", *argv);
    else if ((h = BeginUpdateResource(argv[1], 0)) && EndUpdateResource(h, 0))
        return 0;
    else
        fprintf(stderr, "%s: Begin/End UpdateResource failed -- %u\n",
                *argv, GetLastError());

    return 1;
}

So I'd consider it a gcc issue.

Nevertheless, I still think it's worth enabling app-manifest by default to ensure it's linked, and then if the toolchain has a default manifest then disable it. Or not disabling it, because this 1.5K or so is relatively negligible.

rmyorston commented 12 months ago

It's weird though, the app-manifest is actually 2 bytes smaller than the toolchain manifest (1165/1167), and yet the binary is 1.5K bigger.

Binary sizes always seem to be a multiple of 512 bytes. Alignment of sections, or something. Whatever.

Changing the number of the manifest in resources.rc from 1 to 2 results in two copies of the text rather than one copy and a block of zeroes. Again, whatever.

Thinking about how to proceed from here:

So, if the required manifest (either standard or UTF-8) is enabled in the busybox-w32 configuration and the default manifest isn't installed on Fedora everything will work nicely for Fedora and w64devkit builds.

Builds on MSYS2 will end up with 1.5K of bloat. While I do care about bloat I don't much care about MSYS2, so I can live with that.

avih commented 12 months ago

Changing the number of the manifest in resources.rc from 1 to 2 results in two copies of the text rather than one copy and a block of zeroes. Again, whatever.

Yeah, I believe .exe manifests should have ID 1, and DLL manifests should have ID 2, so I guess gcc doesn't consider ID 2 as overriding to the default exe manifest.

However, I did try changing the language to neutral (adding LANGUAGE 0, 0 at the .rc file), because the default manifest uses language 0.

This did change the language of the custom manifest from 1033 (en-us) to 0, but the file still ended up bigger than it should be, so that didn't help.

Builds on MSYS2 will end up with 1.5K of bloat. While I do care about bloat I don't much care about MSYS2, so I can live with that.

Well, that's not strictly bloat (which typically results from more code etc). It's a gcc issue, which can be fixed in future gcc or in post-build (currently I only know how to do that on windows though), which is a strip-like step to rebuild the .rsrc section without any changes to the data.

But it is still annoying.

There's one more thing to try though, and that's to have a zero-size default-manifest.o (or whatever its name is) at the link path, and hope that it's linked instead of the default manifest.

EDIT: well, that didn't help either. In MSYS2 it's at /mingw64/lib/default-manifest.o, and I touched an empty one at the current dir, compiled the c file and a custom manifest independently, then gcc -L . foo.o rsrc.o -o foo.exe (tried also adding default-manifest.o explicitly), but the resulting file is still bigger than it should, and begin+end UpdateResource still makes it smaller. Whatever.

rmyorston commented 12 months ago

The default configurations now use the supplied manifests. The Unicode build reports the correct OS version and the binary is smaller.

avih commented 12 months ago

Nice.

+ sed -i 's/CONFIG_FEATURE_APP_MANIFEST=y/# CONFIG_FEATURE_APP_MANIFEST is not set/' "$configs"/mingw64u_defconfig

Heh, we need to add a -u NAME or some such thingy to support ensuring that a config is unset :)

Like this maybe (untested)?

diff --git a/scripts/mk_mingw64u_defconfig b/scripts/mk_mingw64u_defconfig
index 760c55a00..dbb6f0d82 100755
--- a/scripts/mk_mingw64u_defconfig
+++ b/scripts/mk_mingw64u_defconfig
@@ -2,10 +2,14 @@

 configs=$(dirname -- "$0")/../configs

-# replace each FOO=bar argument with -e 's/.*FOO.*/FOO=bar/', then sed "$@"
+# update config values in stdin to stdout.
+# FOO=bar arg sets config FOO to bar, -BAZ arg unsets config BAZ
 set_build_opts() {
     for v; do
-        set -- "$@" -e "s/.*${v%%=*}.*/$v/"
+        case $v in
+      -*) set -- "$@" -e "s/.*${v#-}.*/# ${v#-} is not set/";;
+       *) set -- "$@" -e "s/.*${v%%=*}.*/$v/";;
+        esac
         shift
     done
     sed "$@"
@@ -23,6 +27,8 @@ set_build_opts() {
 #   - Full unicode range (U+10FFFF - LAST_SUPPORTED_WCHAR=1114111)

 set_build_opts \
+    -CONFIG_FEATURE_APP_MANIFEST \
+    -CONFIG_FEATURE_TOOLCHAIN_MANIFEST \
     CONFIG_FEATURE_UTF8_MANIFEST=y \
     CONFIG_FEATURE_UTF8_INPUT=y \
     CONFIG_FEATURE_UTF8_OUTPUT=y \
naksyl commented 12 months ago

This still does not report Windows 11 but at least version number is correct.

It looks like MS does not introduce new GUID for Win11 for inclusion in manifest so Win10 and Win11 are distinguishable only by build nuber. Build numbers >= 22000 can be considered as Windows 11, this should work until Win12 Windows 10 builds looks like 1xxxx

Windows release information

Can You consider something like this:

index 357a6fc..ea009da 100644
--- a/win32/uname.c
+++ b/win32/uname.c
@@ -18,7 +18,8 @@ int uname(struct utsname *name)
        os_info.dwOSVersionInfoSize = sizeof(OSVERSIONINFO);

        GetVersionEx(&os_info);
-       sprintf(name->release, "%u.%u", (unsigned int)os_info.dwMajorVersion,
+       sprintf(name->release, "%u.%u",
+                       (unsigned int)os_info.dwBuildNumber >= 22000 ? 11 : (unsigned int)os_info.dwMajorVersion,
                        (unsigned int)os_info.dwMinorVersion);
        sprintf(name->version, "%u", (unsigned int)os_info.dwBuildNumber);

to get Windows 11 recognized ?

$ ./busybox.exe uname -a
Windows_NT lenovo 11.0 22621 x86_64 MS/Windows

but maybe it is not a good idea to do the MS job, whatever

rmyorston commented 12 months ago

Windows 11 is a marketing name, not a version number.

~ $ systeminfo | grep ^OS
OS Name:                   Microsoft Windows 11 Enterprise Evaluation
OS Version:                10.0.22621 N/A Build 22621
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Workstation
OS Build Type:             Multiprocessor Free

There's a similar mismatch in Windows 8.1:

~ $ systeminfo | grep ^OS
OS Name:                   Microsoft Windows 8.1 Enterprise Evaluation
OS Version:                6.3.9600 N/A Build 9600
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Workstation
OS Build Type:             Multiprocessor Free

It's not unusual for the marketing department to become detached from reality. I recall the same sort of thing with SunOS/Solaris.

naksyl commented 12 months ago

Understood, thanks for explanation

ale5000-git commented 12 months ago

The way to get the real Windows version is here: https://dennisbabkin.com/blog/?t=how-to-tell-the-real-version-of-windows-your-app-is-running-on#ver_string It is also implemented in Open-Shell: https://github.com/Open-Shell/Open-Shell-Menu/commit/7f6b7229f6ed30854482c18878abba8b7b200316

avih commented 9 months ago

I think TOOLCHAIN_MANIFEST remains unused?

rmyorston commented 9 months ago

I think TOOLCHAIN_MANIFEST remains unused?

It isn't used in any of the default configurations, but it's available to anyone who needs it for a non-default configuration.

avih commented 9 months ago

it's available to anyone who needs it for a non-default configuration.

Used how? I don't think I see any files which use this value...

rmyorston commented 9 months ago

Setting TOOLCHAIN_MANIFEST prevents the inclusion of either of the supplied manifests.

avih commented 9 months ago

Oh, I guess it works because it's part of a "choice" in Config.in, and it's technically the "do nothing" option alternative to the app/utf8 manifest options...