Linker error on Windows when building an application that links to hmatrix

[obadz]

Using stackage/lts-3.8 with hmatrix manually set to 0.17.0.1. hmatrix itself builds fine, but my application won't link. I get a bunch of such errors:

Z:\.stack-work\install\i386-windows\lts-3.8\7.10.2\lib\i386-windows-ghc-7.10.2\hmatrix-0.17.0.1-BkLqp7XxMCtFB862GgyGAx/libHShmatrix-0.17.0.1-BkLqp7XxMCtFB862GgyGAx.a(vector-aux.o):vector-aux.c:(.text+0x5a52): undefined reference to `nrand48'
collect2: ld returned 1 exit status

The missing symbols are: random, srandom, nrand48

Any ideas on how to fix?

I get the same errors whether I use OpenBLAS 0.2.14 (32bit) or Gilberto Camara's gsl-lapack-windows.zip.

[obadz]

// cc @maxc01 @varosi

[obadz]

I've gotten this down to a small repro:

Main.hs:

{-# OPTIONS_GHC -Wall #-}

module Main (main) where

import Numeric.LinearAlgebra (Matrix)

-- data MyType = MyType (Matrix Double) -- works fine!
-- data MyType = MyType (Matrix Double) deriving Show -- link error
data MyType = MyType (Matrix Double) deriving Eq -- link error

main = putStrLn "This world is hell."

test.cabal:

name:                test
version:             0.1.0.0
build-type:          Simple
cabal-version:       >=1.10

executable test
  main-is:             Main.hs
  build-depends:       base >=4.8 && <4.9
                     , hmatrix
  default-language:    Haskell2010

stack.yaml:

# Specifies the GHC version and set of packages available (e.g., lts-3.5, nightly-2015-09-21, ghc-7.10.2)
resolver: lts-3.8

# Local packages, usually specified by relative directory name
packages:
- '.'

# Packages to be pulled from upstream that are not in the resolver (e.g., acme-missiles-0.3)
extra-deps:
- hmatrix-0.17.0.1

# Override default flag values for local packages and extra-deps
flags:
    hmatrix:
        openblas: true

# Extra directories used by stack for building
extra-include-dirs: [ "z:/OpenBLAS-v0.2.14-Win32/include" ]
extra-lib-dirs:     [ "z:/OpenBLAS-v0.2.14-Win32/lib"     ]

stack build yields the mentioned link error.

[maxc01]

I made a patch, which should solve this problem.

[obadz]

hi @maxc01,

Thank you so much for your prompt patch! Unfortunately, I'm now getting an error when building hmatrix:

src\Internal\C\vector-aux.c: In function 'urandom':

src\Internal\C\vector-aux.c:969:5:
     error: 'errno_t' undeclared (first use in this function)

src\Internal\C\vector-aux.c:969:5:
     note: each undeclared identifier is reported only once for each function it appears in

src\Internal\C\vector-aux.c:969:13:
     error: expected ';' before 'err'

src\Internal\C\vector-aux.c:970:5:
     error: 'err' undeclared (first use in this function)

src\Internal\C\vector-aux.c:970:5:
     warning: implicit declaration of function 'rand_s'

src\Internal\C\vector-aux.c:975:37:
     error: 'UINT_MAX' undeclared (first use in this function)

src\Internal\C\vector-aux.c:976:1:
     warning: control reaches end of non-void function

[maxc01]

Do you use MinGW or something similar?

[obadz]

I just installed stack 0.1.5.0 from this link: https://github.com/commercialhaskell/stack/releases/download/v0.1.5.0/stack-0.1.5.0-i386-windows.zip

Then stack downloaded everything for me, so not sure if that includes MinGW.

[maxc01]

@obadz Sorry, I omit some implementation details. The gcc shipped with Haskell Platform has different configure option with the gcc of my machine(shipped with Strawberry). That's to say, the gcc on my machine can "see" rand_s but the gcc on your machine cannot "see" rand_s. But rand_s indeed exist, it just cannot be find by your gcc. You can just change the urandom function from

#define _CRT_RAND_S
inline double urandom() {
    unsigned int number;
    int err;
    err = rand_s(&number);
    if (err!=0) {
        printf("something wrong\n");
        return -1;
    }
    return (double)number / (double)UINT_MAX;
}

to

inline double urandom() {
    return rand() / (double)RAND_MAX;
}

it will pass compiling, but it is not a good idea.

@albertoruiz Do you think we should stick to rand_s or just use rand on Windows, there is a trade-off. Using rand_s is more secure, but some gcc will complain implicit declaration, using rand is more portable, but rand is really a bad random generator.

[albertoruiz]

@maxc01 I think that we should use the most portable implementation. We can give a warning that it is a bad generator or not thread-safe, and recommend using for example mwc-random (https://github.com/albertoruiz/hmatrix/pull/148#issuecomment-139899353). I will add a small example program to the examples folder.

[obadz]

@maxc01, thanks again for your help, this (https://github.com/obadz/hmatrix/commit/db5fe25c28e6a67eed9fbc65412110480086641b) indeed works.

Another unrelated question: do you have a recommendation as to which is best, OpenBLAS, or gsl-lapack-windows.zip?

Finally, seems like the OpenBLAS binary refers to libgfortran-3.dll but does not ship it. Is that expected?

[varosi]

It is shipped in a separate archive for mingw as I remember. See on the download page of OpenBLAS site. But even if you have it - hmatrix have linker errors when build from subhask. I'll post some related links soon.

[varosi]

I have successfully built hmatrix alone with MinGHC from command-prompt, and not from MSYS or similar.

[varosi]

http://sourceforge.net/projects/openblas/files/v0.2.14/ mingw64_dll.zip

[maxc01]

Sorry, @obadz I have no idea which one is best. If you are on 32bit windows, as @varosi said, you can use the 32bit version, mingw32_dll.zip.

[varosi]

Do someone have success with Windows build?

[maxc01]

using Haskell Platform with the gcc shipped with Haskell Platform, windows build will fail using Haskell Platform with the gcc installed by msys2, windows build will success using Haskell Platform with the gcc shipped with Strawberry, windows build will success

if someone build failed on windows, check gcc -v in windows cmd first, and if there are several gcc installed on windows, make sure the gcc which enabled more options comes first in the PATH environment variable

[obadz]

@varosi, I was able to build https://github.com/obadz/hmatrix/commit/db5fe25c28e6a67eed9fbc65412110480086641b with stack build.

[varosi]

@obadz, build it alone or as dependency of another project? On my site it is built alone correctly, but when it is linked to subhask for example it fails to link.

[obadz]

@varosi, builds as a dependency to an application for me (using my branch, not master).

[varosi]

@obadz, Is it possible to give more information about your setup (which GHC, which MinGHC/Platform Haskell, which MSYS2/CygWin/etc.)? From comments above I see that you are using Stack 0.1.5.0 possibly running from Command Prompt and not from MSYS2 or similar shell. What "gcc --version" is telling you?

And the second, is it possible for you to try to install "subhask" library (that have hmatrix dependency) on your setup?

[obadz]

ghc-7.10.2 No platform Haskell, I used stack 0.1.5.0 with lts/3.8 msys2-20150512

packages:

Win32 2.3.1.0
abstract-deque 0.3
abstract-par 0.3.3
aeson 0.8.0.2
aeson-pretty 0.7.2
appar 0.1.4
array 0.5.1.0
attoparsec 0.12.1.6
auto-update 0.1.2.2
base <unknown>
binary 0.7.5.0
blaze-builder 0.4.0.1
byteorder 1.0.4
bytestring 0.10.6.0
bytestring-builder 0.10.6.0.0
case-insensitive 1.2.0.5
cereal 0.4.1.1
cmdargs 0.10.13
conduit 1.2.5
conduit-extra 1.1.9.1
containers 0.5.6.2
deepseq 1.4.1.1
directory 1.2.2.0
dlist 0.7.1.2
erf 2.0.0.0
exceptions 0.8.0.2
extra 1.4.2
filepath 1.4.0.0
ghc-prim <unknown>
hashable 1.2.3.3
hex 0.1.2
hmatrix 0.17.0.2
http-types 0.8.6
http2 1.0.4
integer-gmp <unknown>
iproute 1.5.0
lifted-base 0.2.3.6
math-functions 0.1.5.2
mmorph 1.0.4
monad-control 1.0.0.4
monad-par 0.3.4.7
monad-par-extras 0.3.3
mtl 2.2.1
mwc-random 0.13.3.2
network 2.6.2.1
old-locale 1.0.0.7
parallel 3.2.0.6
primitive 0.6.1.0
process 1.2.3.0
random 1.1
resourcet 1.1.6
scientific 0.3.4.0
simple-sendfile 0.2.21
split 0.2.2
statistics 0.13.2.3
stm 2.4.4
storable-complex 0.2.2
streaming-commons 0.1.14.2
syb 0.5.1
template-haskell <unknown>
text 1.2.1.3
time 1.5.0.1
transformers 0.4.2.0
transformers-base 0.4.4
transformers-compat 0.4.0.4
unix-compat 0.4.1.4
unordered-containers 0.2.5.1
utf8-string 1.0.1.1
vault 0.3.0.4
vector 0.10.12.3
vector-algorithms 0.7.0.1
vector-binary-instances 0.2.1.0
vector-th-unbox 0.2.1.2
wai 3.0.4.0
warp 3.1.3.1
word8 0.1.2
zlib 0.5.4.2

[varosi]

Thanks for the information! I'll try your configuration on clean machine.

[varosi]

1. So you build it from MSYS2 console?
2. What version of MSYS2 do you use? I saw that you use 32bit MSYS2.
3. Could you paste here your last stack.yaml that your hmatrix is build against?

[obadz]

I don't know what msys2 is. I think stack installed msys2-20150512 for me. I built from cmd console simply typing stack build

[varosi]

Do you building it alone or as a dependency? I have no success with building hmatrix, because of missing atanh symbol.

[varosi]

An interesting compatability table: http://stackoverflow.com/questions/15539116/atanh-arc-hyperbolic-tangent-function-missing-in-ms-visual-c

[mvoidex]

I've spent much time trying to fix 'unknown symbol' (_dgesvd_, ___muldc3, _random etc). With @maxc01 fix I was able to install hmatrix on Windows:

1. Install mingw from https://sourceforge.net/projects/mingw-w64/files/ and add it to PATH (mingw with GHC doesn't contain fortran compiler)
2. git clone https://github.com/xianyi/OpenBLAS
3. make install PREFIX=d:\Programs\openblas (running from powershell works well)
4. git clone https://github.com/maxc01/hmatrix.git
5. cd hmatrix\packages\base
6. cabal install --flags=openblas --extra-lib-dirs=D:\Programs\openblas\lib --extra-include-dirs=D:\Programs\openblas\include
7. place libopenblas.dll, libgcc_s...dll (in my case libgcc_s_dw2-1 from mingw), libquadmath-0.dll in PATH

8. run ghci -llibopenblas -llibgcc_s_dw2-1 -llibquadmath-0 -lmsvcr120

   λ> import Numeric.LinearAlgebra λ> matrix 2 [1..6] (3><2) [ 1.0, 2.0 , 3.0, 4.0 , 5.0, 6.0 ]

[lemmih]

After a long struggle, I found that using 'miniconda' to install openblas (libblas + lapack will not work in ghci) to be the easiest option for me. It's important to install the mingw version of openblas and it requires renaming libgfortran-3.dll to libgfortran.dll.

For exact details, see my CI build script: https://github.com/reanimate/reanimate/blob/master/.azure/azure-windows-template.yml#L28