Build on Windows error -> `can't load .so/.DLL for: libopenblas.dll`

[freeman42x]

I applied the hmatrix commits for stack and build on windows found in https://github.com/varosi/hmatrix/tree/feature/base-stack over the hmatrix 0.16.1.5 which is required as a subhask dependency. When I build only hmatrix it builds without any error.

Adding the altered hmatrix package above to the subhask package yaml configuration yields the following error:

$ stack build
subhask-0.1.1.0: build

--  While building package subhask-0.1.1.0 using:
      C:\\Users\\Razvan\\AppData\\Local\\Programs\\stack\\i386-windows\\ghc-7.10.2\\bin\\runhaskell.exe -package=Cabal-1.22.4.0 -clear-package-db -global-package-db -package-db=C:\Users\Razvan\AppData\Roaming\stack\snapshots\i386-windows\lts-3.3\7.10.2\pkgdb\ C:\Users\Razvan\AppData\Local\Temp\stack148160\Setup.hs --builddir=.stack-work\dist\i386-windows\Cabal-1.22.4.0\ build --ghc-options -hpcdir .stack-work\dist\i386-windows\Cabal-1.22.4.0\hpc\.hpc\
    Process exited with code: ExitFailure 1
    Logs have been written to: D:\Sources\subhask\.stack-work\logs\subhask-0.1.1.0.log

    Building subhask-0.1.1.0...
    Preprocessing library subhask-0.1.1.0...
    [ 6 of 36] Compiling SubHask.Mutable  ( src\SubHask\Mutable.hs, .stack-work\dist\i386-windows\Cabal-1.22.4.0\build\SubHask\Mutable.o )
    <command line>: can't load .so/.DLL for: libopenblas.dll (addDLL: could not load DLL)
    ghc.exe: libopenblas: The specified module could not be found.

The modified subhask yaml file used for building is:

flags:
  hmatrix:
    openblas: true
packages:
- '.'
- location: D:\Sources\hmatrix-varosi
  subdirs:
  - packages\base
extra-deps:
    [ gamma-0.9.0.2
    , continued-fractions-0.9.1.1
    , converge-0.1.0.1
    ]
resolver: lts-3.3

extra-include-dirs:
- D:\Libs\OpenBLAS\OpenBLAS-v0.2.14-Win64-int64\include
extra-lib-dirs:
- D:\Libs\OpenBLAS\OpenBLAS-v0.2.14-Win64-int64\lib

Could instead have passed the extra setting by command line as mentioned in: https://github.com/varosi/hmatrix/blob/feature/base-stack/INSTALL.md

[mikeizbicki]

Hmm...

I suspect that the problem is still in how hmatrix is installed. SubHask doesn't use openblas directly at all. It only accesses it through hmatrix.

I'm not very familiar with either Windows or stack. Is there a chance that stack is not using the fixed version of hmatrix you have installed?

[varosi]

@razvan-panda You need to have libopenblas.dll into your PATH to work.

[freeman42x]

@varosi I had libopenblas.dll in the PATH under /c/WINDOWS/system32. Checked with echo $PATH in the MINGW64 console where I ran the stack build (the shell that comes with latest git for windows 2.5.0).

[freeman42x]

Upgraded stack from 1.2.* to latest 0.1.4.1 and getting the same error. No idea why this is happening as it built ok when building hmatrix alone.

[freeman42x]

This might have something to do with dynamic / enable shared options:

https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/using-shared-libs.html

[freeman42x]

Possibly related: https://ghc.haskell.org/trac/ghc/ticket/8618

or

https://ghc.haskell.org/trac/ghc/ticket/3242

[varosi]

@razvan-panda: I tried right now to copy libopenblas.dll to root of subhask and it stopped to give that message about the DLL. But now it gives different message:

ghc.exe: unable to load package `hmatrix-0.17.0.1'

ghc.exe: D:\Projects\workspace\subhask.cabal-sandbox\x86_64-windows-ghc-7.10.2\hmatrix-0.17.0.1-BKZgcz6cUJt3k7Bf3waJkr\HShmatrix-0.17.0.1-BKZgcz6cUJt3k7Bf3waJkr.o: unknown symbol `__muldc3'

with GHC 7.10.2 x64

[varosi]

I found that this has to do with GCC: http://lists.mcs.anl.gov/pipermail/mpich-discuss/2012-August/012920.html

May be this OpenBLAS is built with GCC and this is the problem. I have tried do get DLLs (mingw64) that are needed for libopenblas.dll into subhask root folder: http://sourceforge.net/projects/openblas/files/v0.2.14/

But without success.

[varosi]

__muldc3 symbol is found in LIBGCC_S_SEH-1.DLL library.

[freeman42x]

@varosi Thank you for the investigation. I am not sure how to approach figuring this out. Tried copying libopenblas.dll into subhask root and the error message did not change for me, same error when copying in the build directory subhask\.stack-work\dist\i386-windows\Cabal-1.22.4.0.

[varosi]

@razvan-panda You are welcome! I think somehow it doesn't load your DLL by three possible reasons:

1. Cannot found it - Windows is searching first in current folder where you're executing executable. In this situation it is cabal. So you have to put it in current working directory;
2. It is not with the correct instruction set - x86 or x64. So you could try with both versions. OpenBLAS has even three of it;
3. It could not find dependencies of libopenblas.dll. Which are from mingw archive that is located here: https://github.com/mikeizbicki/subhask/issues/22#issuecomment-139890899

[varosi]

http://comments.gmane.org/gmane.linux.lib.musl.general/4007

Have more information. I think that the problem comes from OpenBLAS library itself.

[freeman42x]

@varosi I placed the DLL (same architecture as the one which worked for building only hmatrix by itself) in the subhask root folder and built using stack and it did not see it (haven't tried with cabal-install).

[varosi]

@razvan-panda, I have changed this line in hmatrix.cabal:

extra-libraries: libopenblas, libgcc_s_seh-1, libgfortran-3, libquadmath-0

and now it giving me next linker error:

ghc.exe: D:\Projects\workspace\HL.stack-work\install\x86_64-windows\lts-3.3\7.10.2\lib\x86_64-windows-ghc-7.10.2\hmatrix-0.17.0.1-BKZgcz6cUJt3k7Bf3waJkr\HShmatrix-0.17.0.1-BKZgcz6cUJt3k7Bf3waJkr.o: unknown symbol `atanh'

It is connected with GCC StdLib and math functions, I think.

[varosi]

May be the problem is in OpenBLAS Windows distribution itself... It's just too much time wasted because of BLAS/LAPACK stuff.

May be radical solution like this better: https://github.com/albertoruiz/hmatrix/pull/159

Or use some pure Haskell solution.

[varosi]

hm, I think that if the person that is built OpenBLAS for Windows could provide us with DLL that have math functions in it, this could help a lot.

[varosi]

I tried to comment math functions, so almost all math functions are missing in DLLs.