Windows compatibility

[charlesgregory]

So turns out htslib can be built for windows. I attempted to build dhtslib on windows and got errors for the following:

..\..\AppData\Local\dub\packages\dhtslib-0.10.1\dhtslib\source\dhtslib\htslib\bgzf.d(162,13): Error: undefined identifier ssize_t, did you mean alias size_t?

ssize_t is not defined for windows in the d library. We can probably alias it for windows.

[charlesgregory]

After a couple of edits to dhtslib, compiling htslib with mingw64, a lot of searching on how to get a .lib file from .dll, and including all necessary dlls, I did manage to compile a program with dhtslib on windows and get it to run. So definitely possible, just tedious. See #39

[charlesgregory]

Just to record it somewhere. Here are the steps to replicate this on an x86_64 machine:

• Install mingw from http://www.msys2.org/. Or you can use Git Bash for Windows with the SDK.
• Open mingw64 or gitbash sdk shell
• run pacman -S --noconfirm --needed base-devel mingw-w64-x86_64-toolchain mingw-w64-x86_64-zlib mingw-w64-x86_64-bzip2 mingw-w64-x86_64-xz mingw-w64-x86_64-curl
• clone dhtslib and do git submodule init; git submodule update
• move to htslib folder and run autoheader && autoconf && ./configure && make -j4 && make install. This should generate hts-2.dll.
• run gendef hts-2.dll
• run lib /DEF:hts-2.def /OUT:hts.lib /MACHINE:X64
• copy hts-2.dll and hts.lib to dhtslib folder (hts-2.dll actually needs to be wherever the final program is if you're building dhtslib as a dependency)
• It should now compile, but it will likely not run.
• The fun part: find all the associated dlls neccessary for htslib and place them with hts-2.dll (all of mine were under C:\Program Files\Git\mingw64\bin)

If all of this was done correctly, it should run.

[jblachly]

@charlesgregory can https://github.com/blachlylab/dhtslib/commit/3c657950c2ae3192d69d2c797553df982d3b9899 close this?

[charlesgregory]

Just a list of dlls required (copy to dhtslib folder) on my machine (found in my mingw64\bin folder) to run unittests:

hts-2.dll
libbrotlicommon.dll
libbrotlidec.dll
libbz2-1.dll
libcrypto-1_1-x64.dll
libcurl-4.dll
libiconv-2.dll
libidn2-0.dll
libintl-8.dll
liblzma-5.dll
libnghttp2-14.dll
libssh2-1.dll
libssl-1_1-x64.dll
libunistring-2.dll
zlib1.dll

[charlesgregory]

Updates as my initial instructions were a bit wrong. If using MinGW as your environment you need pacman -S --noconfirm --needed base-devel mingw-w64-x86_64-toolchain mingw-w64-x86_64-zlib mingw-w64-x86_64-bzip2 mingw-w64-x86_64-xz mingw-w64-x86_64-curl. If using msys2 you need to use pacman -S gcc git zlib-devel libbz2-devel liblzma-devel ncurses-devel libcurl-devel && pacman -Fy. Then autoreconf -i && ./configure && make -j4 && make install. Use pacman -F gendef.exe to find a gendef exe and then /mingw64/bin/gendef.exe hts-3.dll. For msys2 dlltool.exe -l hts.lib -d hts-3.def hts-3.dll, for mingw lib /DEF:hts-2.def /OUT:hts.lib /MACHINE:X64. And then follow the rest of the previous instructions to get the needed dlls.

Did this for msys2

cp /usr/bin/msys-bz2-1.dll /usr/bin/msys-curl-4.dll /usr/bin/msys-crypto-1.1.dll /usr/bin/msys-lzma-5.dll /usr/bin/msys-2.0.dll /usr/bin/msys-z.dll /usr/bin/msys-brotlidec-1.dll /usr/bin/msys-idn2-0.dll /usr/bin/msys-gssapi-3.dll /usr/bin/msys-nghttp2-14.dll /usr/bin/msys-psl-5.dll /usr/bin/msys-ssh2-1.dll /usr/bin/msys-ssl-1.1.dll /usr/bin/msys-zstd-1.dll /usr/bin/msys-brotlicommon-1.dll /usr/bin/msys-asn1-8.dll /usr/bin/msys-com_err-1.dll /usr/bin/msys-heimntlm-0.dll /usr/bin/msys-heimbase-1.dll /usr/bin/msys-hcrypto-4.dll /usr/bin/msys-krb5-26.dll /usr/bin/msys-roken-18.dll /usr/bin/msys-iconv-2.dll /usr/bin/msys-intl-8.dll /usr/bin/msys-unistring-2.dll /usr/bin/msys-wind-0.dll /usr/bin/msys-gcc_s-seh-1.dll /usr/bin/msys-hx509-5.dll /usr/bin/msys-crypt-0.dll /usr/bin/msys-sqlite3-0.dll ../

Compiles but is broken so unsure what is wrong. Tested only msys2 so unsure if mingw is broken.

[charlesgregory]

I seem to have hit an interesting snag. htslib can be built on windows via mingw64 or msys2 and used to build dhtslib but it doesn't pass unittests. This is because dmd and ldc by default use the MSVC toolchain. So your malloc/free calls internally in a mingw64 built htslib use mingw's heap, however Dlang is using MSVC's heap. So any free calls in the D code will crash when freeing htslib data. You also can't realloc htslib data from D (which we do).

This also explains why building with msys2 doesn't work either for basically the same reason, htslib and dhtslib are being built from two different toolchains.

In order to resolve this, we will have to have either a d compiler with support for compiling with mingw64 or htslib will have to become fully MSVC compatible. Currently dmd has no mingw support, ldc's is limited to mingw32, gdc maybe has newly found mingw64 support but finding documentation is hard. Meanwhile htslib appears to be moving to be (or is already) MSVC compatible, but I cannot find information on how to build it as it is a very different toolchain that I have little experience with.

This is a very low priority issue, but I would love the opportunity to allow native windows development (that doesn't require WSL).

[jblachly]

This is a very low priority issue,

say no more