Closed noughtmare closed 4 years ago
@noughtmare Looks like you need to build from source in order to be able to use the bindings:
For general usage of ArrayFire, we recommend the installer binaries. However, if you wish to dev on ArrayFire, you will need to build the source code.
https://github.com/arrayfire/arrayfire/wiki#build-instructions
I think that "dev on ArrayFire" means modifying the ArrayFire library itself. I just want to use it, so the binaries should be enough (the binaries include the /opt/arrayfire/lib64
and /opt/arrayfire/include
directories). But I will try compiling from source to be sure.
I have finished building from source and I still get the same error.
Have you tried using the flags it tells you to? --extra-lib-dirs
and/or --extra-include-dirs
? You shouldnt need to build from source at all.
Yes
This fixes it for me:
diff --git a/arrayfire.cabal b/arrayfire.cabal
index 1f53125..8c4e1f9 100644
--- a/arrayfire.cabal
+++ b/arrayfire.cabal
@@ -96,7 +96,7 @@ library
extra-lib-dirs:
/opt/arrayfire/lib
ld-options:
- -Wl -rpath /opt/arrayfire/lib
+ -Wl,-rpath /opt/arrayfire/lib
Glad that works. Instead of a patch, it should be passed to cabal as --ghc-options (since we can't assume how arrayfire was installed on the system).
I think it is just an error in your .cabal file. AFAIK there should always be a comma after -Wl
and not a space.
Ah, makes sense. Looking back I just realised that those linker flags are already there. Perhaps they shouldn't be.
I think @dmjio and myself haven't noticed because we use nix. I think we should add a travis setup for Ubuntu
Actually, opt is a standard enough place to let that be for non-NixOS. We should just document that, and apply the fix you gave @noughtmare. I'm on mobile right now so I can't do that.
@noughtmare nice catch, strange that error didn't manifest on OSX for me (when I wasn't using nix). I can make a PR for this.
I had a similar problem with ArrayFire installed to a custom location. We (me and @dmjio ) are working on figuring out the cause of it. Passing custom location using --extra-lib-dirs
and --extra-include-dirs
didn't resolve the issue.
I had a similar problem with ArrayFire installed to a custom location. We (me and @dmjio ) are working on figuring out the cause of it. Passing custom location using
--extra-lib-dirs
and--extra-include-dirs
didn't resolve the issue.
what was the actual error?
stack install --extra-lib-dirs=$AF_PATH/lib64 --extra-include-dirs=$AF_PATH/include arrayfire
arrayfire> configure
arrayfire> [1 of 2] Compiling Main ( /tmp/stack-e44f53084a6bc144/arrayfire-0.1.0.0/Setup.hs, /tmp/stack-e44f53084a6bc144/arrayfire-0.1.0.0/.stack-work/dist/x86_64-linux-tinfo6/Cabal-2.4.0.1/setup/Main.o )
arrayfire> [2 of 2] Compiling StackSetupShim ( /home/pradeep/.stack/setup-exe-src/setup-shim-mPHDZzAJ.hs, /tmp/stack-e44f53084a6bc144/arrayfire-0.1.0.0/.stack-work/dist/x86_64-linux-tinfo6/Cabal-2.4.0.1/setup/StackSetupShim.o )
arrayfire> Linking /tmp/stack-e44f53084a6bc144/arrayfire-0.1.0.0/.stack-work/dist/x86_64-linux-tinfo6/Cabal-2.4.0.1/setup/setup ...
arrayfire> Configuring arrayfire-0.1.0.0...
arrayfire> setup: Missing dependency on a foreign library:
arrayfire> * Missing (or bad) C library: af
arrayfire> This problem can usually be solved by installing the system package that
arrayfire> provides this library (you may need the "-dev" version). If the library is
arrayfire> already installed but in a non-standard location then you can use the flags
arrayfire> --extra-include-dirs= and --extra-lib-dirs= to specify where it is.If the
arrayfire> library file does exist, it may contain errors that are caught by the C
arrayfire> compiler at the preprocessing stage. In this case you can re-run configure
arrayfire> with the verbosity flag -v3 to see the error messages.
arrayfire>
-- While building package arrayfire-0.1.0.0 using:
/tmp/stack-e44f53084a6bc144/arrayfire-0.1.0.0/.stack-work/dist/x86_64-linux-tinfo6/Cabal-2.4.0.1/setup/setup --builddir=.stack-work/dist/x86_64-linux-tinfo6/Cabal-2.4.0.1 configure --user --package-db=clear --package-db=global --package-db=/home/pradeep/.stack/snapshots/x86_64-linux-tinfo6/8096d5c493ca1014d2da8250c11e63731bfe2fe6565556a6aa8a501eadfda3c2/8.6.5/pkgdb --libdir=/home/pradeep/.stack/snapshots/x86_64-linux-tinfo6/8096d5c493ca1014d2da8250c11e63731bfe2fe6565556a6aa8a501eadfda3c2/8.6.5/lib --bindir=/home/pradeep/.stack/snapshots/x86_64-linux-tinfo6/8096d5c493ca1014d2da8250c11e63731bfe2fe6565556a6aa8a501eadfda3c2/8.6.5/bin --datadir=/home/pradeep/.stack/snapshots/x86_64-linux-tinfo6/8096d5c493ca1014d2da8250c11e63731bfe2fe6565556a6aa8a501eadfda3c2/8.6.5/share --libexecdir=/home/pradeep/.stack/snapshots/x86_64-linux-tinfo6/8096d5c493ca1014d2da8250c11e63731bfe2fe6565556a6aa8a501eadfda3c2/8.6.5/libexec --sysconfdir=/home/pradeep/.stack/snapshots/x86_64-linux-tinfo6/8096d5c493ca1014d2da8250c11e63731bfe2fe6565556a6aa8a501eadfda3c2/8.6.5/etc --docdir=/home/pradeep/.stack/snapshots/x86_64-linux-tinfo6/8096d5c493ca1014d2da8250c11e63731bfe2fe6565556a6aa8a501eadfda3c2/8.6.5/doc/arrayfire-0.1.0.0 --htmldir=/home/pradeep/.stack/snapshots/x86_64-linux-tinfo6/8096d5c493ca1014d2da8250c11e63731bfe2fe6565556a6aa8a501eadfda3c2/8.6.5/doc/arrayfire-0.1.0.0 --haddockdir=/home/pradeep/.stack/snapshots/x86_64-linux-tinfo6/8096d5c493ca1014d2da8250c11e63731bfe2fe6565556a6aa8a501eadfda3c2/8.6.5/doc/arrayfire-0.1.0.0 --dependency=Cabal=Cabal-2.4.1.0-3JC906oWQ73Ah7bYkIcI4J --dependency=base=base-4.12.0.0 --dependency=cabal-doctest=cabal-doctest-1.0.8-40Ym5zAcLNVCMmwzciX9Qk --dependency=directory=directory-1.3.3.0 --dependency=filepath=filepath-1.4.2.1 --dependency=parsec=parsec-3.1.14.0-8t8UKeFouAo5hdmomuIjPj --dependency=text=text-1.2.3.1 --dependency=vector=vector-0.12.0.3-LfvlcMFJAcY18uD1Y2O5Ig -f-disable-default-paths --extra-include-dirs=/home/pradeep/gitroot/ArrayFireWorkspace/worktrees/v3.6/build/pkg/include --extra-lib-dirs=/home/pradeep/gitroot/ArrayFireWorkspace/worktrees/v3.6/build/pkg/lib64 --exact-configuration --ghc-option=-fhide-source-paths
Process exited with code: ExitFailure 1
@9prady9 I think your problem is that the line:
Still links to the wrong directory. So it will fail with something like (if you pass -v3 to cabal somehow):
/usr/bin/gcc returned ExitFailure 1 with error message:
gcc: error: /opt/arrayfire/lib: No such file or directory
I have tried building with that line removed and that still works, so maybe that line should just be removed completely.
Another way to fix it for your case is to pass the disable-default-paths
flag to arrayfire.
@9prady9 I think your problem is that the line:
Still links to the wrong directory. So it will fail with something like (if you pass -v3 to cabal somehow):
/usr/bin/gcc returned ExitFailure 1 with error message: gcc: error: /opt/arrayfire/lib: No such file or directory
I have tried building with that line removed and that still works, so maybe that line should just be removed completely.
Another way to fix it for your case is to pass the
disable-default-paths
flag to arrayfire.
If that is the case, why I do I still see the paths I am passing via command line in the command output I shared earlier ?
Also, when I pass extra options via command line, the following appears in the command output
-f-disable-default-paths --extra-include-dirs=/home/pradeep/gitroot/ArrayFireWorkspace/worktrees/v3.6/build/pkg/include --extra-lib-dirs=/home/pradeep/gitroot/ArrayFireWorkspace/worktrees/v3.6/build/pkg/lib64
no default paths are being disabled if I am not wrong. Please correct me if I am understanding it incorrectly.
If that is the case, why I do I still see the paths I am passing via command line in the command output I shared earlier ?
The correct paths do get passed to gcc, but it will also pass the wrong paths which do not exist.
-f-disable-default-paths
I think this means that the flag is disabled (I think this is explaned here in the cabal documentation), but you want to enable it. In the stack documentation I read that that is done using:
--flag arrayfire:disable-default-paths
Adding that additional flag did carry the build process much further, the following command did finish successfully. @noughtmare thank you.
stack install --flag arrayfire:disable-default-paths \
--extra-lib-dirs=$AF_PATH/lib64 --extra-include-dirs=$AF_PATH/include arrayfire
I think it worked as it copied couple of binaries into $HOME/.local/bin
, gen and main, I was able to run main
. Is it possible to place them somewhere local in a build sort of directory ? Or is this the normal to place haskell binaries in .local/bin
?
@dmjio however nix-shell --run test-runner
command fails. It's probably the same issue as above but with nix-shell or whatever it does. I am not familiar with nix-shell. I think the README guide could definitely use more improvements on the set of instructions required to use arrayfire-haskell via all three routes (as per my understanding) : cabal, stack , nix-shell.
@dmjio what does gen
executable do ? It is writing some files in the location where I ran the executable.
@9prady9
Is it possible to place them somewhere local in a build sort of directory ? Or is this the normal to place haskell binaries in .local/bin ?
stack build
will put the executables in an obfuscated local (I mean local to the package not literally .local
) folder somewhere in .stack-work
(I think it prints the exact location to the command line). The executables can then be run with stack exec main
for example.
true, it is printing the path as $HOME/.local/bin
.
I was concerned that this path doesn't always necessarily is present in PATH
environment variable and the user needs to add it (although a one time thing) to run the executable via stack exec
command.
Is there a way to control where stack generates the executable and lookup for executable ? something like a configuration option ?
There is a local bin path flag: https://docs.haskellstack.org/en/stable/yaml_configuration/#local-bin-path
stack install --local-bin-path="~/.bin"
should work
@9prady9 hmmm, nix-shell --run test-runner
works for me on OSX and NixOS. Can you paste the error you're receiving and I can diagnose.
@dmjio what does
gen
executable do ? It is writing some files in the location where I ran the executable.
@9prady9 gen
is used to lex / parse the C headers into Haskell FFI declarations. Should only be ran when upgrading versions.
It is same error message as earlier one with stack command.
On Wed, 6 Nov, 2019, 08:50 David Johnson, notifications@github.com wrote:
@9prady9 https://github.com/9prady9 hmmm, nix-shell --run test-runner works for me on OSX and NixOS. Can you paste the error you're receiving and I can diagnose.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/arrayfire/arrayfire-haskell/issues/10?email_source=notifications&email_token=AAY6OOWT6XGK5OI3A43JUKDQSIZWHA5CNFSM4JIS4TM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDFDXKA#issuecomment-550124456, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAY6OORPQDEXOPXWTIFBPV3QSIZWHANCNFSM4JIS4TMQ .
Understood, thanks for the clarification
On Wed, 6 Nov, 2019, 09:12 David Johnson, notifications@github.com wrote:
@dmjio https://github.com/dmjio what does gen executable do ? It is writing some files in the location where I ran the executable.
gen is used to lex / parse the C headers in to Haskell FFI declarations. Should only be ran when upgrading versions.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/arrayfire/arrayfire-haskell/issues/10?email_source=notifications&email_token=AAY6OOVXCFO7R6DBH436Y6DQSI4IVA5CNFSM4JIS4TM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDFE4YY#issuecomment-550129251, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAY6OOV4CJW365BJ6TCA4I3QSI4IVANCNFSM4JIS4TMQ .
when I run on OSX, I get:
λ Davids-MacBook-Pro arrayfire-haskell → λ git master* → nix-shell --run test-runner
Resolving dependencies...
Configuring arrayfire-0.2.0.0...
Preprocessing library for arrayfire-0.2.0.0..
Building library for arrayfire-0.2.0.0..
Preprocessing test suite 'test' for arrayfire-0.2.0.0..
Building test suite 'test' for arrayfire-0.2.0.0..
ArrayFire.Algorithm
Algorithm tests
Should sum a scalar
Should sum a vector
Should sum a default value to replace NaN
Should product a scalar
Should product a vector
Should product a default value to replace NaN
Should take the minimum element of a vector
Should find if all elements are true along dimension
Should find if any elements are true along dimension
Should get count of all elements
Should get sum all elements
Should get sum all elements
Should product all elements in an Array
Should product all elements in an Array
Should find minimum value of an Array
Should find maximum value of an Array
ArrayFire.Arith
Arith tests
Should negate scalar value
Should negate a vector
Should add two scalar arrays
Should add two scalar bool arrays
Should subtract two scalar arrays
Should multiply two scalar arrays
Should divide two scalar arrays
Should add two matrices
Should take cubed root
Should take square root
Should lt Array
Should lte Array
Should gte Array
Should gt Array
Should eq Array
Should and Array
Should and Array
Should or Array
Should not Array
Should bitwise and array
Should bitwise or array
Should bitwise xor array
Should bitwise shift left an array
Should cast an array
Should find the minimum of two arrays
Should find the max of two arrays
Should take the clamp of 3 arrays
Should check if an array has positive or negative infinities
Should check if an array has any NaN values
Should check if an array has any Zero values
ArrayFire.Array
Array tests
Should perform Array tests
...
ArrayFire.Signal
Signal spec
Should do FFT in place
Should do FFT
ArrayFire.Sparse
Sparse spec
Should create a sparse array
ArrayFire.Statistics
Statistics spec
Should find the mean
Should find the weighted-mean
Should find the variance
Should find the weighted variance
Should find the standard deviation
Should find the covariance
Should find the median
Should find the mean of all elements across all dimensions
Should find the weighted mean of all elements across all dimensions
Should find the variance of all elements across all dimensions
Should find the weighted variance of all elements across all dimensions
Should find the stdev of all elements across all dimensions
Should find the median of all elements across all dimensions
Should find the correlation coefficient
Should find the top k elements
ArrayFire.Util
Util spec
Should get size of
Should get version
Should get revision
Should save / read array
ArrayFire.Vision
Vision spec
Should construct Features for fast feature detection
Finished in 6.2213 seconds
122 examples, 0 failures
what OS are you on?
Arch Linux i can get you the Haskell version a bit later. I am not in front of the PC right now.
On Wed, 6 Nov, 2019, 10:36 David Johnson, notifications@github.com wrote:
when I run on OSX, I get:
λ Davids-MacBook-Pro arrayfire-haskell → λ git master* → ns --run test-runner
Resolving dependencies...
Configuring arrayfire-0.2.0.0...
Preprocessing library for arrayfire-0.2.0.0..
Building library for arrayfire-0.2.0.0..
Preprocessing test suite 'test' for arrayfire-0.2.0.0..
Building test suite 'test' for arrayfire-0.2.0.0..
ArrayFire.Algorithm
Algorithm tests
Should sum a scalar Should sum a vector Should sum a default value to replace NaN Should product a scalar Should product a vector Should product a default value to replace NaN Should take the minimum element of a vector Should find if all elements are true along dimension Should find if any elements are true along dimension Should get count of all elements Should get sum all elements Should get sum all elements Should product all elements in an Array Should product all elements in an Array Should find minimum value of an Array Should find maximum value of an Array
ArrayFire.Arith
Arith tests
Should negate scalar value Should negate a vector Should add two scalar arrays Should add two scalar bool arrays Should subtract two scalar arrays Should multiply two scalar arrays Should divide two scalar arrays Should add two matrices Should take cubed root Should take square root Should lt Array Should lte Array Should gte Array Should gt Array Should eq Array Should and Array Should and Array Should or Array Should not Array Should bitwise and array Should bitwise or array Should bitwise xor array Should bitwise shift left an array Should cast an array Should find the minimum of two arrays Should find the max of two arrays Should take the clamp of 3 arrays Should check if an array has positive or negative infinities Should check if an array has any NaN values Should check if an array has any Zero values
ArrayFire.Array
Array tests
Should perform Array tests
In function af::dim4 verifyDims(const unsigned int, const dim_t *const)
In file src/api/c/data.cpp:38
Invalid dimension for argument 2
Expected: dims[i] >= 1
Should fail to create 0 dimension arrays
In function af::dim4 verifyDims(const unsigned int, const dim_t *const)
In file src/api/c/data.cpp:38
Invalid dimension for argument 2
Expected: dims[i] >= 1
Should fail to create 0 length arrays
In function af::dim4 verifyDims(const unsigned int, const dim_t *const)
In file src/api/c/data.cpp:38
Invalid dimension for argument 2
Expected: dims[i] >= 1
Should fail to create 0 length arrays w/ 0 dimensions Should create a column vector Should create a row vector Should create a vector Should create a vector Should copy an array Should modify manual eval flag Should return the number of elements Should create a scalar array Should get number of dims specified Should get value of dims specified Should test Sparsity Should make a Bit array Should make an integer array Should make a Floating array Should make a Complex array Should make a Real array Should make a Double precision array Should make a Single precision array Should make a Real floating array Should get reference count Should convert an array to a list
ArrayFire.Backend
Backend spec
Should get backend count Should get available backends Should set backend to CPU
ArrayFire.BLAS
BLAS spec
Should matmul two matrices Should dot product two vectors Should produce scalar dot product between two vectors as a Complex number Should take the transpose of a matrix Should take the transpose of a matrix in place
ArrayFire.Data
Data tests
Should create constant Array
ArrayFire.Device
Algorithm tests
ArrayFire v3.6.4 (OpenCL, 64-bit Mac OSX, build 1b8030c5)
[0] APPLE: AMD Radeon Pro 555X Compute Engine, 4096 MB
-1- APPLE: Intel(R) UHD Graphics 630, 1536 MB
Should show device info Should show device init Should get info string Should get device Should get and set device
ArrayFire.Features
Feautures tests
Should get features number an array
ArrayFire.Graphics
Graphics tests
Should create window
ArrayFire.Image
Image tests
Should test if Image I/O is available
ArrayFire.Index
Index spec
Should index into an array
ArrayFire.LAPACK
LAPACK spec
Should have LAPACK available Should perform svd Should perform svd in place Should perform lu Should perform qr Should get determinant of Double Should calculate inverse
ArrayFire.Random
Random engine spec
Should create random engine Should set random engine Should set and get seed
ArrayFire.Signal
Signal spec
Should do FFT in place Should do FFT
ArrayFire.Sparse
Sparse spec
Should create a sparse array
ArrayFire.Statistics
Statistics spec
Should find the mean Should find the weighted-mean Should find the variance Should find the weighted variance Should find the standard deviation Should find the covariance Should find the median Should find the mean of all elements across all dimensions Should find the weighted mean of all elements across all dimensions Should find the variance of all elements across all dimensions Should find the weighted variance of all elements across all dimensions Should find the stdev of all elements across all dimensions Should find the median of all elements across all dimensions Should find the correlation coefficient Should find the top k elements
ArrayFire.Util
Util spec
Should get size of Should get version Should get revision Should save / read array
ArrayFire.Vision
Vision spec
Should construct Features for fast feature detection
Finished in 6.2213 seconds
122 examples, 0 failures
what OS are you on?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/arrayfire/arrayfire-haskell/issues/10?email_source=notifications&email_token=AAY6OORDV4XCZY4VIZNQFVTQSJGF5A5CNFSM4JIS4TM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDFJBAY#issuecomment-550146179, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAY6OOVBSNABX4QM3GLPPBTQSJGF5ANCNFSM4JIS4TMQ .
I think the issue is that this ld-option
should be removed.
-Wl,-rpath /opt/arrayfire/lib
It's presumptuous to assume we can always link with that directory even when the user is using the binary distribution (since it could be lib
or lib64
), and the failure always ends in compilation failure when its not found. Instead, we can add both
extra-lib-dirs:
/opt/arrayfire/lib64
/opt/arrayfire/lib
And it will simply warn if one of the directories is missing w/o compilation failure. This should make a much better experience on initial install.
What about the custom installation paths with nix-shell ? Please check my earlier messages for details.
Just in case anyone is looking for a simple workaround for this problem you can just use a symbolic link:
$ sudo ln -s /opt/arrayfire/lib64 /opt/arrayfire/lib
$ sudo bash -c 'echo /opt/arrayfire/lib > /etc/ld.so.conf.d/arrayfire.conf'
$ sudo ldconfig
@lehins can you confirm if you install arrayfire with the binary installer on Linux that it creates /opt/arrayfire/lib64
and not /opt/arrayfire/lib
(related to #46). Don't have ubuntu handy, and trying to make assumptions out here.
@dmjio That is correct, I can confirm this on Ubuntu.
@lehins awesome! And not to be redundant, but can you also confirm that you did not install from source, but from the binary installer script ?
@dmjio lol. No worries. I did indeed install it from the downloadable Linux installer
@lehins perfect ! then I'm going to go ahead and merge #46, since I think the default locations are accounted for on each platform. People who build from source will have to -f+disable-default-paths
and --extra-lib-dirs=/path/to/arrayfire/lib
and --extra-include-dirs=/path/to/arrayfire/include
@noughtmare @9prady9 @chessai @lehins this issue should be solved in 0.6.0.0
. (no more symbolic link required)
If not, please re-open.
@9prady9 this issue is solved only in the case end-users are using the binary distribution of ArrayFire w/ the Haskell wrapper (not installing from source). I have an issue (and solution) to provide documentation for this case. #48
I have installed arrayfire 3.6.4 from https://arrayfire.com/download/ to
/opt/arrayfire
as described on http://arrayfire.org/docs/installing.htm#Linux.Running
cabal install arrayfire
yields the following error message: