dbry / WavPack

WavPack encode/decode library, command-line programs, and several plugins
BSD 3-Clause "New" or "Revised" License
371 stars 67 forks source link

Add support for Windows on ARM64 #115

Closed carlo-bramini closed 8 months ago

ArminiusTux commented 8 months ago

A great addition @carlo-bramini , however I failed in compiling your v5.4.0 ARM tree.

grafik

wavpack.sln was used with VS 2022 (version 17.9.2):

> Build started at 18:51...
> 1>------ Skipped Build: Project: winamp_lng, Configuration: Release ARM64 ------
> 1>Project not selected to build for this solution configuration 
> 2>------ Build started: Project: wvtag, Configuration: Release ARM64 ------
> 3>------ Build started: Project: wavpackdll, Configuration: Release ARM64 ------
> 4>------ Skipped Build: Project: audition, Configuration: Release ARM64 ------
> 4>Project not selected to build for this solution configuration 
> 5>------ Skipped Build: Project: winamp, Configuration: Release ARM64 ------
> 5>Project not selected to build for this solution configuration 
> 6>------ Build started: Project: wvgain, Configuration: Release ARM64 ------
> 7>------ Build started: Project: wvunpack, Configuration: Release ARM64 ------
> 8>------ Build started: Project: wavpack, Configuration: Release ARM64 ------
> 7>LINK : fatal error LNK1246: '/DYNAMICBASE:NO' not compatible with 'ARM64' target machine; link without '/DYNAMICBASE:NO'
> 7>Done building project "wvunpack.vcxproj" -- FAILED.
> 8>LINK : fatal error LNK1246: '/DYNAMICBASE:NO' not compatible with 'ARM64' target machine; link without '/DYNAMICBASE:NO'
> 8>Done building project "wavpack.vcxproj" -- FAILED.
> 3>LINK : fatal error LNK1246: '/DYNAMICBASE:NO' not compatible with 'ARM64' target machine; link without '/DYNAMICBASE:NO'
> 3>Done building project "wavpackdll.vcxproj" -- FAILED.
> 6>LINK : fatal error LNK1246: '/DYNAMICBASE:NO' not compatible with 'ARM64' target machine; link without '/DYNAMICBASE:NO'
> 6>Done building project "wvgain.vcxproj" -- FAILED.
> 2>LINK : fatal error LNK1246: '/DYNAMICBASE:NO' not compatible with 'ARM64' target machine; link without '/DYNAMICBASE:NO'
> 2>Done building project "wvtag.vcxproj" -- FAILED.
> ========== Build: 0 succeeded, 5 failed, 1 up-to-date, 3 skipped ==========
> ========== Build completed at 18:51 and took 12,247 seconds ==========

Would you mind updating your changes against the new v5.7.0 source, please?

As for compiling WavPack for Windows on ARM in general ... ... it works with Cmake+MSVC & also MSYS2 (Cmake/Clang/Ninja).

The multithreading gains are fantastic, thanks @dbry for this addition!
Please consider turning this welcome feature ON by default in a future version (v6?).
dbry commented 8 months ago

The reason I did not pull this in initially is because it seemed to break my VS 2019 setup, although I forget the details now. And of course I have no way to test a Windows ARM64 build (still not even really sure what that is). But I'm happy to bring something in if it doesn't break my setup and others claim it works. Sorry @carlo-bramini I didn't post before... :(

As for making multithreading the default @ArminiusTux, yes it would certainly make sense in a future release, if not the next. I just thought I'd give it a little bake time to make sure it didn't break in some situations, but so far it's been really stable.

ArminiusTux commented 8 months ago

Hello and sorry for the late reply.

Sadly the changes don't hit the target yet: ``` Build started at 14:04... 1>------ Skipped Build: Project: winamp_lng, Configuration: Release ARM64 ------ 1>Project not selected to build for this solution configuration 2>------ Build started: Project: libwavpack, Configuration: Release ARM64 ------ 2>common_utils.c 2>decorr_utils.c 2>entropy_utils.c 2>extra1.c 2>extra2.c 2>open_filename.c 2>open_legacy.c 2>open_raw.c 2>open_utils.c 2>pack.c 2>pack_dns.c 2>pack_dsd.c 2>pack_floats.c 2>pack_utils.c 2>read_words.c 2>tags.c 2>tag_utils.c 2>unpack.c 2>unpack3.c 2>unpack3_open.c 2>Generating Code... 2>Compiling... 2>unpack3_seek.c 2>unpack_dsd.c 2>unpack_floats.c 2>unpack_seek.c 2>unpack_utils.c 2>write_words.c 2>Generating Code... 2>libwavpack.vcxproj -> C:\CODING\WavPack\ARM64\Release\libwavpack.lib 3>------ Build started: Project: wvtest, Configuration: Release x64 ------ 4>------ Build started: Project: wvtag, Configuration: Release ARM64 ------ 5>------ Build started: Project: wavpackdll, Configuration: Release ARM64 ------ 6>------ Skipped Build: Project: audition, Configuration: Release ARM64 ------ 6>Project not selected to build for this solution configuration 7>------ Skipped Build: Project: winamp, Configuration: Release ARM64 ------ 7>Project not selected to build for this solution configuration 8>------ Build started: Project: wvgain, Configuration: Release ARM64 ------ 9>------ Build started: Project: wvunpack, Configuration: Release ARM64 ------ 10>------ Build started: Project: wavpack, Configuration: Release ARM64 ------ 3>md5.c 3>wvtest.c 4>import_id3.c 8>utils.c 5>dummy.c 10>aiff.c 3>Generating Code... 10>caff.c 10>dsdiff.c 10>dsf.c 10>import_id3.c 9>aiff_write.c 9>caff_write.c 4>utils.c 8>win32_unicode_support.c 9>dsdiff_write.c 9>dsf_write.c 9>md5.c 9>riff_write.c 9>utils.c 10>md5.c 10>riff.c 10>utils.c 8>wvgain.c 4>win32_unicode_support.c 9>wave64_write.c 8>Generating Code... 9>win32_unicode_support.c 10>wave64.c 10>wavpack.c 4>wvtag.c 9>wvunpack.c 4>Generating Code... 10>win32_unicode_support.c 9>Generating Code... 5>LINK : fatal error LNK1246: '/DYNAMICBASE:NO' not compatible with 'ARM64' target machine; link without '/DYNAMICBASE:NO' 5>Done building project "wavpackdll.vcxproj" -- FAILED. 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackOpenFileInputEx referenced in function decode_thread 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackOpenFileInput referenced in function seeking_test 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackGetMode referenced in function seeking_test 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackGetQualifyMode referenced in function seeking_test 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackUnpackSamples referenced in function decode_thread 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackGetNumSamples64 referenced in function seeking_test 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackGetNumErrors referenced in function decode_thread 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackSeekSample referenced in function seeking_test 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackSeekSample64 referenced in function seeking_test 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackCloseFile referenced in function decode_thread 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackGetBytesPerSample referenced in function decode_thread 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackGetNumChannels referenced in function decode_thread 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackGetMD5Sum referenced in function seeking_test 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackOpenFileOutput referenced in function run_test 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackSetConfiguration64 referenced in function run_test 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackStoreMD5Sum referenced in function run_test 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackPackInit referenced in function run_test 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackPackSamples referenced in function run_test 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackFlushSamples referenced in function run_test 3>wvtest.obj : error LNK2019: unresolved external symbol WavpackGetLibraryVersionString referenced in function main 3>C:\CODING\WavPack\ARM64\Release\libwavpack.lib : warning LNK4272: library machine type 'ARM64' conflicts with target machine type 'x64' 3>C:\CODING\WavPack\x64\Release\wvtest.exe : fatal error LNK1120: 20 unresolved externals 3>Done building project "wvtest.vcxproj" -- FAILED. 10>Generating Code... 8>LINK : fatal error LNK1246: '/DYNAMICBASE:NO' not compatible with 'ARM64' target machine; link without '/DYNAMICBASE:NO' 4>LINK : fatal error LNK1246: '/DYNAMICBASE:NO' not compatible with 'ARM64' target machine; link without '/DYNAMICBASE:NO' 8>Done building project "wvgain.vcxproj" -- FAILED. 4>Done building project "wvtag.vcxproj" -- FAILED. 9>LINK : fatal error LNK1246: '/DYNAMICBASE:NO' not compatible with 'ARM64' target machine; link without '/DYNAMICBASE:NO' 9>Done building project "wvunpack.vcxproj" -- FAILED. 10>LINK : fatal error LNK1246: '/DYNAMICBASE:NO' not compatible with 'ARM64' target machine; link without '/DYNAMICBASE:NO' 10>Done building project "wavpack.vcxproj" -- FAILED. ========== Build: 1 succeeded, 6 failed, 0 up-to-date, 3 skipped ========== ========== Build completed at 14:04 and took 30,034 seconds ========== ```

I managed to update all the *.vcxproj files, which leads to a successful ARM64 compilation.

😁 ``` Build started at 18:10... 1>------ Skipped Build: Project: winamp_lng, Configuration: Release ARM64 ------ 1>Project not selected to build for this solution configuration 2>------ Build started: Project: libwavpack, Configuration: Release ARM64 ------ 2>common_utils.c 2>decorr_utils.c 2>entropy_utils.c 2>extra1.c 2>extra2.c 2>open_filename.c 2>open_legacy.c 2>open_raw.c 2>open_utils.c 2>pack.c 2>pack_dns.c 2>pack_dsd.c 2>pack_floats.c 2>pack_utils.c 2>read_words.c 2>tags.c 2>tag_utils.c 2>unpack.c 2>unpack3.c 2>unpack3_open.c 2>Generating Code... 2>Compiling... 2>unpack3_seek.c 2>unpack_dsd.c 2>unpack_floats.c 2>unpack_seek.c 2>unpack_utils.c 2>write_words.c 2>Generating Code... 2>libwavpack.vcxproj -> C:\CODING\WavPack-MODIFIED\ARM64\Release\libwavpack.lib 3>------ Build started: Project: wvtest, Configuration: Release ARM64 ------ 4>------ Build started: Project: wvtag, Configuration: Release ARM64 ------ 5>------ Build started: Project: wavpackdll, Configuration: Release ARM64 ------ 6>------ Skipped Build: Project: audition, Configuration: Release ARM64 ------ 6>Project not selected to build for this solution configuration 7>------ Skipped Build: Project: winamp, Configuration: Release ARM64 ------ 7>Project not selected to build for this solution configuration 8>------ Build started: Project: wvgain, Configuration: Release ARM64 ------ 9>------ Build started: Project: wvunpack, Configuration: Release ARM64 ------ 10>------ Build started: Project: wavpack, Configuration: Release ARM64 ------ 3>md5.c 3>wvtest.c 4>import_id3.c 5>dummy.c 8>utils.c 3>Generating Code... 9>aiff_write.c 9>caff_write.c 9>dsdiff_write.c 9>dsf_write.c 10>aiff.c 9>md5.c 9>riff_write.c 10>caff.c 9>utils.c 10>dsdiff.c 4>utils.c 10>dsf.c 8>win32_unicode_support.c 10>import_id3.c 9>wave64_write.c 4>win32_unicode_support.c 10>md5.c 9>win32_unicode_support.c 10>riff.c 8>wvgain.c 10>utils.c 8>Generating Code... 4>wvtag.c 9>wvunpack.c 10>wave64.c 10>wavpack.c 4>Generating Code... 9>Generating Code... 10>win32_unicode_support.c 10>Generating Code... 5> Creating library C:\CODING\WavPack-MODIFIED\ARM64\Release\wavpackdll.lib and object C:\CODING\WavPack-MODIFIED\ARM64\Release\wavpackdll.exp 9>wvunpack.vcxproj -> C:\CODING\WavPack-MODIFIED\ARM64\Release\wvunpack.exe 8>wvgain.vcxproj -> C:\CODING\WavPack-MODIFIED\ARM64\Release\wvgain.exe 3>wvtest.vcxproj -> C:\CODING\WavPack-MODIFIED\ARM64\Release\wvtest.exe 4>wvtag.vcxproj -> C:\CODING\WavPack-MODIFIED\ARM64\Release\wvtag.exe 5>wavpackdll.vcxproj -> C:\CODING\WavPack-MODIFIED\ARM64\Release\wavpackdll.dll 10>wavpack.vcxproj -> C:\CODING\WavPack-MODIFIED\ARM64\Release\wavpack.exe ========== Build: 7 succeeded, 0 failed, 0 up-to-date, 3 skipped ========== ========== Build completed at 18:10 and took 27,292 seconds ========== ```

However the result is unuseable as all EXEs hang (Debug & Release). It turned out that the wavpackdll.dll is faulty, cause if replaced (by a cmake+msvc shared dll build) the executables are fine.

Any suggestions?

carlo-bramini commented 8 months ago

Hello! after lot of time, I tried to build again wavpack for WOA and actually you are right, it seems that /DINAMICBASE isn't supported on ARM devices. See: https://learn.microsoft.com/en-us/cpp/build/reference/dynamicbase?view=msvc-170 immagine I'm sorry, I don't understand how I could not seen it in the past, but hopefully this can be easily fixed into the settings of the project and, if I understood correctly, you already did when doing your successful ARM64 compilation.

dbry commented 8 months ago

No problem. But if I understand @ArminiusTux correctly, he was able to fix the compilation, but the DLL does not work (hangs).

@carlo-bramini were you actually able to get this to run on an ARM64 machine back when you originally created the patch? If not then including this might be premature. Unfortunately I have no such hardware or expertise.

ArminiusTux commented 8 months ago

Good evening to you both,

indeed compilation works but libwavpack (the shared wavpackdll.dll) is most likely causing the hang. grafik

I also took the clean v5.7.0 source (without the ARM additions) and got this upon cloning the configuration profile (x64->ARM64): grafik

Maybe those assembler optimizations (for x86 & x64 only) are somehow being applied to the ARM64 build despite being defined for exclusion in the libwavpack.vcxproj. Is there a neat way to drop them altogether (just for testing)?

Unfortunately I have no such hardware or expertise.

Well @dbry the need for real metal or silicon, is no longer a must. 😉

Happy Easter!

dbry commented 7 months ago

Happy Easter!

I too thought about it being the ASM code, but I looked in your build dump and they're not listed, and there's no ASM involved in the WavpackGetLibraryVersionString() function which seems to be hanging in your example. My guess is that it's an ABI issue, although I would normally assume that would crash rather than hang. Kind of a mystery.

If I had a spare week I would dig into it, but I'm short on time and, to tell the truth, interest (especially when there are other ways to build this). Unless someone comes to the rescue I might pull this out before anyone gets hurt. :smiley:

ArminiusTux commented 7 months ago

By all means @dbry , let's rewind this cassette 📼 and remove the ARM64 VisualStudio additions for the time being.

carlo-bramini commented 7 months ago

Perhaps, it is also true that all files for VisualStudio may be simply deleted. Unless there are particular reasons for keeping them, the IDE should be perfectly able to open the CMakeLists.txt file and re-create the solution and the project files for the platform you want, i686, x64 and ARM64, making shared or static libraries and so on. EDIT: this way just needs to update the README file with the build instructions for Windows when using MSVC, actually. MinGW, MSYS2 and CYGWIN can just follow the same behaviour of unix-like systems.

dbry commented 7 months ago

Okay, for now I've pulled out the ARM64 mods. We can look into this later, but for now Cmake+MSVC will do the job, or I use Cmake+mingw on Linux to build everything, even the Cool Edit and Winamp plugins (although those of course do not have ARM64 equivalents).

Thanks both of you!