support building on x64

[kusma]

Here's a rework of #30, where I did a few changes:

• Split up the main commit into several commits
• Fixed a TODO
• Cleaned up the code a bit
• Fixed the indentation problens
• Added appveyor-builds of x64
• Rewored the size_t -> int patch to be smaller
• Wrote more comprehensive commit messages

I kept the authorship for the patches I copied verbatim. The other patches, I wrote on top.

[kusma]

One thing I'm wondering is what kinds of dependencies the x64 PlayerTest binary ends up with. I think it's generally OK if we end up with some MSVC runtime dependency for now (and ofc that's a fine requirement for all the plugs etc where the context isn't size-limited) but eventually I'd like to make sure x64 builds don't depend on anything that wouldn't be shipped in a clean, standard Windows install (standard rule for 64k compos). It might not be an issue with these patches anyways, but we should double-check (if nothing else looking at .dll dependencies is probably a good idea).

Yeah, so @lamogui's motivation was to build the VST, not really to build the player into a 64-bit intro.

And my motivation is porting the code to Linux and macOS, which differs quite a lot in run-time dependencies.

So, this doesn't technically matter to us (yet).

But I'll take my x64 build and dump the import table a bit later today, so we can see what it does, though.

Due to lack of 64k-focues x64 packers (currently, working on that separately!) I wouldn't mind merging this either way, but I'd like to have a better overview beforehand at least.

Yeah. I think knowing exactly how to effectively target x64 in 64k is still kinda unknown, so I think we should take things one step at the time. But I would be very interested in seeing that solved as well!

So yeah, I'll provide the import-table, and we'll see if there's anything scary popping up there.

[kusma]

@yupferris: Oh, shit. Turns out, the MinSizeRel configuration doesn't even build on x64, mainly because msvcrt_old.lib is for x86 So here's the output of "dumpbin /imports Tests\PlayerTest\Release\PlayerTest.exe" instead:

Microsoft (R) COFF/PE Dumper Version 14.16.27032.1
Copyright (C) Microsoft Corporation.  All rights reserved.

Dump of file Tests\PlayerTest\Release\PlayerTest.exe

File Type: EXECUTABLE IMAGE

  Section contains the following imports:

    DSOUND.dll
             140008000 Import Address Table
             140009BC8 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                             Ordinal    11

    KERNEL32.dll
             140008010 Import Address Table
             140009BD8 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                         58B Sleep
                          86 CloseHandle
                         524 SetEvent
                         5E6 WaitForSingleObject
                          BC CreateEventA
                         5E4 WaitForMultipleObjects
                          F2 CreateThread
                         56B SetThreadPriority
                         367 InitializeCriticalSection
                         135 EnterCriticalSection
                         3C0 LeaveCriticalSection
                         111 DeleteCriticalSection
                         21D GetCurrentProcess
                         27E GetModuleHandleW
                         389 IsProcessorFeaturePresent
                         57B SetUnhandledExceptionFilter
                         5BC UnhandledExceptionFilter
                         382 IsDebuggerPresent
                         4E1 RtlVirtualUnwind
                         4DA RtlLookupFunctionEntry
                         4D3 RtlCaptureContext
                         36C InitializeSListHead
                         2F0 GetSystemTimeAsFileTime
                         222 GetCurrentThreadId
                         21E GetCurrentProcessId
                         450 QueryPerformanceCounter
                         59A TerminateProcess

    USER32.dll
             1400080F0 Import Address Table
             140009CB8 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                         121 GetAsyncKeyState
                         158 GetForegroundWindow

    VCRUNTIME140.dll
             140008108 Import Address Table
             140009CD0 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                          3C memcpy
                          36 _purecall
                           1 _CxxThrowException
                          22 __std_exception_destroy
                          21 __std_exception_copy
                           8 __C_specific_handler
                          3E memset

    api-ms-win-crt-stdio-l1-1-0.dll
             140008258 Import Address Table
             140009E20 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                          54 _set_fmode
                          7D fopen
                           3 __stdio_common_vfprintf
                           1 __p__commode
                          92 putchar
                           0 __acrt_iob_func
                          8A fwrite
                          80 fputs
                          74 fclose

    api-ms-win-crt-math-l1-1-0.dll
             140008180 Import Address Table
             140009D48 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                          90 cos
                          C9 fmod
                          FE pow
                         116 sqrt
                          BE floor
                           9 __setusermatherr
                         117 sqrtf

    api-ms-win-crt-heap-l1-1-0.dll
             140008148 Import Address Table
             140009D10 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                          16 _set_new_mode
                          18 free
                          19 malloc
                           8 _callnewh

    api-ms-win-crt-runtime-l1-1-0.dll
             1400081C0 Import Address Table
             140009D88 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                          15 _c_exit
                          16 _cexit
                          42 _set_app_type
                          34 _initialize_onexit_table
                          3C _register_onexit_function
                          1E _crt_atexit
                          67 terminate
                           5 __p___argv
                          18 _configure_narrow_argv
                          3D _register_thread_local_exe_atexit_callback
                           4 __p___argc
                          33 _initialize_narrow_environment
                          28 _get_initial_narrow_environment
                          36 _initterm
                          37 _initterm_e
                          23 _exit
                          55 exit
                          40 _seh_filter_exe

    api-ms-win-crt-locale-l1-1-0.dll
             140008170 Import Address Table
             140009D38 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                           8 _configthreadlocale

  Summary

        2000 .data
        1000 .pdata
        3000 .rdata
        1000 .reloc
        1000 .rsrc
        7000 .text
        1000 _RDATA

While I don't particularly care too much about MinSizeRel on x64, this made me realize that we don't even test MinSizeRel on CI for x86!

[kusma]

Just thought I'd mention it; I gave it a quick stab, and just dropping the msvcrt_old.lib stuff makes MinSizeRel build fine on x64. So that's the only tweak that's needed there.

Adding MinSizeRel to CI is a separate topic, I think. But we should probably build that as well.

[kusma]

Here's the import-table from a MinSizeRel build after fixing:

Microsoft (R) COFF/PE Dumper Version 14.16.27032.1
Copyright (C) Microsoft Corporation.  All rights reserved.

Dump of file Tests\PlayerTest\MinSizeRel\PlayerTest.exe

File Type: EXECUTABLE IMAGE

  Section contains the following imports:

    DSOUND.dll
             140006000 Import Address Table
             140007460 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                             Ordinal    11

    KERNEL32.dll
             140006010 Import Address Table
             140007470 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                         58B Sleep
                         56B SetThreadPriority
                         5E6 WaitForSingleObject
                          F2 CreateThread
                         5E4 WaitForMultipleObjects
                         524 SetEvent
                          86 CloseHandle
                          BC CreateEventA
                         135 EnterCriticalSection
                         3C0 LeaveCriticalSection
                         367 InitializeCriticalSection
                         111 DeleteCriticalSection
                         36C InitializeSListHead
                         4DA RtlLookupFunctionEntry
                         4E1 RtlVirtualUnwind
                         2F0 GetSystemTimeAsFileTime
                         222 GetCurrentThreadId
                         382 IsDebuggerPresent
                         4D3 RtlCaptureContext
                         5BC UnhandledExceptionFilter
                         57B SetUnhandledExceptionFilter
                         389 IsProcessorFeaturePresent
                         27E GetModuleHandleW
                         21E GetCurrentProcessId
                         450 QueryPerformanceCounter

    USER32.dll
             1400060E0 Import Address Table
             140007540 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                         158 GetForegroundWindow
                         121 GetAsyncKeyState

    VCRUNTIME140.dll
             1400060F8 Import Address Table
             140007558 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                           1 _CxxThrowException
                          3E memset
                          3C memcpy
                          21 __std_exception_copy
                           8 __C_specific_handler
                          22 __std_exception_destroy
                          36 _purecall

    api-ms-win-crt-stdio-l1-1-0.dll
             140006230 Import Address Table
             140007690 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                           0 __acrt_iob_func
                           3 __stdio_common_vfprintf
                           1 __p__commode
                          80 fputs
                          8A fwrite
                          74 fclose
                          7D fopen
                          92 putchar
                          54 _set_fmode

    api-ms-win-crt-string-l1-1-0.dll
             140006280 Import Address Table
             1400076E0 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                          86 strcmp

    api-ms-win-crt-heap-l1-1-0.dll
             140006138 Import Address Table
             140007598 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                          19 malloc
                          18 free
                           8 _callnewh
                          16 _set_new_mode

    api-ms-win-crt-runtime-l1-1-0.dll
             140006198 Import Address Table
             1400075F8 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                          15 _c_exit
                          16 _cexit
                          40 _seh_filter_exe
                           5 __p___argv
                          34 _initialize_onexit_table
                          3C _register_onexit_function
                          1E _crt_atexit
                          67 terminate
                          42 _set_app_type
                          18 _configure_narrow_argv
                           4 __p___argc
                          33 _initialize_narrow_environment
                          3D _register_thread_local_exe_atexit_callback
                          36 _initterm
                          23 _exit
                          55 exit
                          37 _initterm_e
                          28 _get_initial_narrow_environment

    api-ms-win-crt-math-l1-1-0.dll
             140006170 Import Address Table
             1400075D0 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                          FE pow
                          C9 fmod
                          90 cos
                           9 __setusermatherr

    api-ms-win-crt-locale-l1-1-0.dll
             140006160 Import Address Table
             1400075C0 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                           8 _configthreadlocale

  Summary

        2000 .data
        1000 .pdata
        2000 .rdata
        5000 .text

So yeah. Pretty much the same, and a lot more than on x86. So I guess checking if we can use a 64-bit version of MSVCRT.dll would be interesting...

[kusma]

Just thought I'd mention it; I gave it a quick stab, and just dropping the msvcrt_old.lib stuff makes MinSizeRel build fine on x64. So that's the only tweak that's needed there.

Adding MinSizeRel to CI is a separate topic, I think. But we should probably build that as well.

BTW, this is exactly what #42 does.

[yupferris]

Right, that's what I was afraid of. If a 64-bit version of MSVCRT.dll exists then it's probably worth looking into; I imagine that being the shortest path, though we'll need to verify for sure.

In any case, I feel more comfortable knowing that it will be an issue and we can ofc tackle that in a separate PR. These changes look good to me as-is (perhaps minus the comment I'd like removed).

[kusma]

@yupferris: I have a patch using a 64-bit MSVCRT.dll on top of this + #42, but it's not quite there yet. I'm currently having problems with a missing mainCRTStartup symbol, which seems to miss from the 64-bit version of MSVCRT.dll. We could also look into if using the newer UCRT-stuff could be enough for us (that's what the import-tables above is about, but it's not done carefully there).

In either case, this gets something that wasn't working before working, so maybe we should merge, and size-optimize as a follow-up? We can see how my work with MSVCRT.dll goes...

[kusma]

This is what I have now:

C:\Users\kusma\source\repos>dumpbin /IMPORTS WaveSabre\build-x64\Tests\PlayerTest\MinSizeRel\PlayerTest.exe
Microsoft (R) COFF/PE Dumper Version 14.25.28612.0
Copyright (C) Microsoft Corporation.  All rights reserved.

Dump of file WaveSabre\build-x64\Tests\PlayerTest\MinSizeRel\PlayerTest.exe

File Type: EXECUTABLE IMAGE

  Section contains the following imports:

    msvcrt.dll
             140005090 Import Address Table
             140005B98 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                          11 ??2@YAPEAX_K@Z
                         2B3 fopen
                         2F6 memset
                         1C1 _purecall
                         2A8 fclose
                         2F4 memcpy
                         2FC printf
                         2FE putchar
                          74 __getmainargs
                         316 strcmp
                         2C3 fwrite
                         2B7 fputs
                          13 ??3@YAXPEAX@Z
                         2FA pow
                         2A0 cos
                         2B2 fmod

    DSOUND.dll
             140005000 Import Address Table
             140005B08 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                             Ordinal    11

    KERNEL32.dll
             140005010 Import Address Table
             140005B18 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                         367 InitializeCriticalSection
                         3C0 LeaveCriticalSection
                         135 EnterCriticalSection
                          BC CreateEventA
                         111 DeleteCriticalSection
                         524 SetEvent
                         5E4 WaitForMultipleObjects
                          F2 CreateThread
                         56B SetThreadPriority
                         58B Sleep
                          86 CloseHandle
                         5E6 WaitForSingleObject

    USER32.dll
             140005078 Import Address Table
             140005B80 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                         121 GetAsyncKeyState
                         158 GetForegroundWindow

  Summary

        2000 .data
        1000 .pdata
        1000 .rdata
        4000 .text

Turns out, the 64-bit version of MSVCRT.dll is missing mainCRTStartup, that part gets statically linked into the exe. But it seems to be relatively easy to bring up enough for PlayerTest.cpp:

extern "C" {
    int __getmainargs(int *argc, char ***argv, char ***env, int dowildcards, int *new_mode);
    int mainCRTStartup()
    {
        int argc;
        char **argv, **env;
        int new_mode = 0;
        __getmainargs(&argc, &argv, &env, 0, &new_mode);
        return main(argc, argv);
    }
}

[kusma]

@yupferris: So, this is what the end result looks like: https://github.com/kusma/WaveSabre/tree/msvcrt-x64

[kusma]

@yupferris: as far as I can tell, this should be good to go. I'll update #42 to use MSVCRT.dll on x64 afterwards.

[kusma]

Anything holding this back?

[yupferris]

Anything holding this back?

Nope, just me being late to respond. Looks good to me!