could not load library "/usr/local/lib/libVkFFTCUDA.so"

[roflmaostc]

Hi,

is it expected that I install libVkFFTCUDA myself? Is there a way to install that as CUDA.jl does it with downloading an Artifact?

julia> using VkFFTCUDA
[ Info: Precompiling VkFFTCUDA [fce8808e-e16a-45a8-8cbe-ffae295c7ea5]
[ Info: Skipping precompilation since __precompile__(false). Importing VkFFTCUDA [fce8808e-e16a-45a8-8cbe-ffae295c7ea5].

julia> p = plan_fft!(xc);
ERROR: UndefVarError: `VKFFT_MAX_FFT_DIMENSIONS` not defined
Stacktrace:
 [1] _make_plan(x::CuArray{…}, region::Tuple{…}, forward::Bool, inplace::Bool; coalesced_memory::Int64, aim_threads::Int64, num_shared_banks::Int64)
   @ VkFFTCUDA ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:316
 [2] _make_plan
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:313 [inlined]
 [3] #_make_tuned_plan#5
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:427 [inlined]
 [4] _make_tuned_plan
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:421 [inlined]
 [5] plan_fft!
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:433 [inlined]
 [6] plan_fft!(X::CuArray{ComplexF32, 3, CUDA.Mem.DeviceBuffer})
   @ FFTW ~/.julia/packages/FFTW/6nZei/src/fft.jl:800
 [7] top-level scope
   @ REPL[20]:1
 [8] top-level scope
   @ ~/.julia/packages/CUDA/htRwP/src/initialization.jl:206
Some type information was truncated. Use `show(err)` to see complete types.

julia> p = plan_fft!(xc, VK);^C

julia> VK^C

julia> VKFFT_MAX_FFT_DIMENSIONS = 3
3

julia> p = plan_fft!(xc);
ERROR: UndefVarError: `VKFFT_MAX_FFT_DIMENSIONS` not defined
Stacktrace:
 [1] _make_plan(x::CuArray{…}, region::Tuple{…}, forward::Bool, inplace::Bool; coalesced_memory::Int64, aim_threads::Int64, num_shared_banks::Int64)
   @ VkFFTCUDA ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:316
 [2] _make_plan
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:313 [inlined]
 [3] #_make_tuned_plan#5
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:427 [inlined]
 [4] _make_tuned_plan
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:421 [inlined]
 [5] plan_fft!
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:433 [inlined]
 [6] plan_fft!(X::CuArray{ComplexF32, 3, CUDA.Mem.DeviceBuffer})
   @ FFTW ~/.julia/packages/FFTW/6nZei/src/fft.jl:800
 [7] top-level scope
   @ REPL[22]:1
 [8] top-level scope
   @ ~/.julia/packages/CUDA/htRwP/src/initialization.jl:206
Some type information was truncated. Use `show(err)` to see complete types.

julia> export VKFFT_MAX_FFT_DIMENSIONS = 3
ERROR: syntax: invalid assignment location "export VKFFT_MAX_FFT_DIMENSIONS" around REPL[23]:1
Stacktrace:
 [1] top-level scope
   @ REPL[23]:1
 [2] top-level scope
   @ ~/.julia/packages/CUDA/htRwP/src/initialization.jl:206

julia> p = plan_fft!(xc);
ERROR: UndefVarError: `VKFFT_MAX_FFT_DIMENSIONS` not defined
Stacktrace:
 [1] _make_plan(x::CuArray{…}, region::Tuple{…}, forward::Bool, inplace::Bool; coalesced_memory::Int64, aim_threads::Int64, num_shared_banks::Int64)
   @ VkFFTCUDA ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:316
 [2] _make_plan
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:313 [inlined]
 [3] #_make_tuned_plan#5
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:427 [inlined]
 [4] _make_tuned_plan
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:421 [inlined]
 [5] plan_fft!
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:433 [inlined]
 [6] plan_fft!(X::CuArray{ComplexF32, 3, CUDA.Mem.DeviceBuffer})
   @ FFTW ~/.julia/packages/FFTW/6nZei/src/fft.jl:800
 [7] top-level scope
   @ REPL[24]:1
 [8] top-level scope
   @ ~/.julia/packages/CUDA/htRwP/src/initialization.jl:206
Some type information was truncated. Use `show(err)` to see complete types.

julia> VkFFT VKFFT_MAX_FFT_DIMENSIONS = 3
VkFFTCUDA
VkFFTPlan
julia> VkFFTCUDA.VKFFT_MAX_FFT_DIMENSIONS = 3
3

julia> p = plan_fft!(xc);
ERROR: could not load library "/usr/local/lib/libVkFFTCUDA.so"
/usr/local/lib/libVkFFTCUDA.so: cannot open shared object file: No such file or directory
Stacktrace:
 [1] _new_config(fft_dim::Int64, buffer_dim::Int64, dims::Vector{…}, omit_dims::Vector{…}, num_batches::Int64, coalesced_memory::Int64, aim_threads::Int64, num_shared_banks::Int64, forward::Bool, double_precision::Bool, inplace::Bool)
   @ VkFFTCUDA ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:275
 [2] _make_plan(x::CuArray{…}, region::Tuple{…}, forward::Bool, inplace::Bool; coalesced_memory::Int64, aim_threads::Int64, num_shared_banks::Int64)
   @ VkFFTCUDA ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:329
 [3] _make_plan
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:313 [inlined]
 [4] #_make_tuned_plan#5
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:427 [inlined]
 [5] _make_tuned_plan
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:421 [inlined]
 [6] plan_fft!
   @ ~/.julia/packages/VkFFTCUDA/aSyIl/src/VkFFTCUDA.jl:433 [inlined]
 [7] plan_fft!(X::CuArray{ComplexF32, 3, CUDA.Mem.DeviceBuffer})
   @ FFTW ~/.julia/packages/FFTW/6nZei/src/fft.jl:800
 [8] top-level scope
   @ REPL[26]:1
 [9] top-level scope
   @ ~/.julia/packages/CUDA/htRwP/src/initialization.jl:206
Some type information was truncated. Use `show(err)` to see complete types.
``

[PaulVirally]

In principle, it installs libVkFFTCUDA for you as part of the building process (see here), but I have to admit it's been very tough getting this to build properly so the current solution is a bit hacky and might not work perfectly well.

When you installed VkFFTCUDA, did it prompt you for your password at some point? Because the current building solution is so terrible, it prompts you for your password, but the prompt is not very clear sometimes. Its possible you missed that and didn't put your password in and the install failed?

[roflmaostc]

The package manager still installs 0.1.0 though.

(@main) pkg> activate --temp
  Activating new project at `/tmp/jl_VOanA4`

(jl_VOanA4) pkg> add CUDA, VkFFTCUDA
   Resolving package versions...
    Updating `/tmp/jl_VOanA4/Project.toml`
  [052768ef] + CUDA v5.2.0
  [fce8808e] + VkFFTCUDA v0.1.0
    Updating `/tmp/jl_VOanA4/Manifest.toml`
  [621f4979] + AbstractFFTs v1.5.0
  [79e6a3ab] + Adapt v4.0.1
  [a9b6321e] + Atomix v0.1.0
⌃ [ab4f0b2a] + BFloat16s v0.4.2
  [fa961155] + CEnum v0.5.0
  [052768ef] + CUDA v5.2.0
  [1af6417a] + CUDA_Runtime_Discovery v0.2.3
  [3da002f7] + ColorTypes v0.11.4
  [5ae59095] + Colors v0.12.10
  [34da2185] + Compat v4.12.0
  [a8cc5b0e] + Crayons v4.1.1
  [9a962f9c] + DataAPI v1.16.0
  [a93c6f00] + DataFrames v1.6.1
  [864edb3b] + DataStructures v0.18.16
  [e2d170a0] + DataValueInterfaces v1.0.0
  [e2ba6199] + ExprTools v0.1.10
  [53c48c17] + FixedPointNumbers v0.8.4
  [0c68f7d7] + GPUArrays v10.0.2
  [46192b85] + GPUArraysCore v0.1.6
  [61eb1bfa] + GPUCompiler v0.25.0
  [842dd82b] + InlineStrings v1.4.0
  [41ab1584] + InvertedIndices v1.3.0
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [692b3bcd] + JLLWrappers v1.5.0
  [63c18a36] + KernelAbstractions v0.9.16
  [929cbde3] + LLVM v6.5.0
  [8b046642] + LLVMLoopInfo v1.0.0
  [b964fa9f] + LaTeXStrings v1.3.1
  [1914dd2f] + MacroTools v0.5.13
  [e1d29d7a] + Missings v1.1.0
  [5da4648a] + NVTX v0.3.4
  [bac558e1] + OrderedCollections v1.6.3
  [69de0a69] + Parsers v2.8.1
  [2dfb63ee] + PooledArrays v1.4.3
  [aea7be01] + PrecompileTools v1.2.0
  [21216c6a] + Preferences v1.4.1
  [08abe8d2] + PrettyTables v2.3.1
  [74087812] + Random123 v1.6.2
  [e6cf234a] + RandomNumbers v1.5.3
  [189a3867] + Reexport v1.2.2
  [ae029012] + Requires v1.3.0
  [6c6a2e73] + Scratch v1.2.1
  [91c51154] + SentinelArrays v1.4.1
  [a2af1166] + SortingAlgorithms v1.2.1
  [90137ffa] + StaticArrays v1.9.2
  [1e83bf80] + StaticArraysCore v1.4.2
  [892a3eda] + StringManipulation v0.3.4
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.11.1
  [a759f4b9] + TimerOutputs v0.5.23
  [013be700] + UnsafeAtomics v0.2.1
  [d80eeb9a] + UnsafeAtomicsLLVM v0.1.3
  [fce8808e] + VkFFTCUDA v0.1.0
  [4ee394cb] + CUDA_Driver_jll v0.7.0+1
  [76a88914] + CUDA_Runtime_jll v0.11.1+0
  [9c1d0b0a] + JuliaNVTXCallbacks_jll v0.2.1+0
  [dad2f222] + LLVMExtra_jll v0.0.28+0
  [e98f9f5b] + NVTX_jll v3.1.0+2
  [0dad84c5] + ArgTools v1.1.1
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching
  [9fa8497b] + Future
  [b77e0a4c] + InteractiveUtils
  [4af54fe1] + LazyArtifacts
  [b27032c2] + LibCURL v0.6.4
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.10.0
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays v1.10.0
  [10745b16] + Statistics v1.10.0
  [fa267f1f] + TOML v1.0.3
  [a4e569a6] + Tar v1.10.0
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v1.1.0+0
  [deac9b47] + LibCURL_jll v8.4.0+0
  [e37daf67] + LibGit2_jll v1.6.4+0
  [29816b5a] + LibSSH2_jll v1.11.0+1
  [c8ffd9c3] + MbedTLS_jll v2.28.2+1
  [14a3606d] + MozillaCACerts_jll v2023.1.10
  [4536629a] + OpenBLAS_jll v0.3.23+4
  [bea87d4a] + SuiteSparse_jll v7.2.1+1
  [83775a58] + Zlib_jll v1.2.13+1
  [8e850b90] + libblastrampoline_jll v5.8.0+1
  [8e850ede] + nghttp2_jll v1.52.0+1
  [3f19e933] + p7zip_jll v17.4.0+2
        Info Packages marked with ⌃ have new versions available and may be upgradable.

julia> using VkFFTCUDA
[ Info: Precompiling VkFFTCUDA [fce8808e-e16a-45a8-8cbe-ffae295c7ea5]
[ Info: Skipping precompilation since __precompile__(false). Importing VkFFTCUDA [fce8808e-e16a-45a8-8cbe-ffae295c7ea5].

[PaulVirally]

Yeah I'm not sure why it's still installing 0.1.0 and not 0.2.0. I pushed 0.2.0 less than an hour ago, maybe the registry needs some time to update correctly?

But anyways I noticed that the build step does not always run. I am not sure what causes the build step to run or not when you ]add VkFTTCUDA, but to force it to run, you can do ]build VkFFTCUDA. That should hopefully solve your issue

[roflmaostc]

Did not really do the job:

(jl_VBJsJU) pkg> build VkFFTCUDA
    Building VkFFTCUDA → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/beaf9abe83bb9d38288c6efb8cb5a6ea6608e2b4/build.log`

(jl_VBJsJU) pkg> 

[PaulVirally]

What is in ~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/beaf9abe83bb9d38288c6efb8cb5a6ea6608e2b4/build.log?

[roflmaostc]

Does not exist for me

[roflmaostc]

Also 0.2.0 fails somehow:

(jl_iB3Jcf) pkg> build VkFFTCUDA
    Building VkFFTCUDA → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/beaf9abe83bb9d38288c6efb8cb5a6ea6608e2b4/build.log`

(jl_iB3Jcf) pkg> st
Status `/tmp/jl_iB3Jcf/Project.toml`
  [621f4979] AbstractFFTs v1.5.0
  [052768ef] CUDA v5.2.0
  [fce8808e] VkFFTCUDA v0.2.0

(jl_iB3Jcf) pkg> 

[roflmaostc]

Ah wait, wrong computer :small_airplane: nvcc not found. Please install CUDA to use VkFFTCUDA.jl

[roflmaostc]

I only installed a graphics driver for CUDA.

All other dependencies CUDA.jl takes care for me. CUDA.jl works on the same machine. Is there a way you could use the same lib as they do?

[PaulVirally]

There is a "proper" way of adding binary artifacts to a Julia package, using BinaryBuilder.jl. I tried to get it to work, but I never figured out how to get nvcc in the sandbox environments that BinaryBuilder provides you with.

If you have any ideas for fixing this, that would be awesome! But in the meantime, the fastest way to get VkFFTCUDA running would be to install the CUDA toolkit

[roflmaostc]

@maleadt can maybe help? He set up all the Cuda environment, I guess :)

[maleadt]

There is a "proper" way of adding binary artifacts to a Julia package, using BinaryBuilder.jl. I tried to get it to work, but I never figured out how to get nvcc in the sandbox environments that BinaryBuilder provides you with.

Use CUDA.required_dependencies as defined in Yggdrasil, this gives you a build-time dependency on CUDA_SDK_jll providing nvcc etc, and a run-time dependency on CUDA_Runtime_jll, all correctly versioned. Have a look at other build recipes in Yggdrasil, there's plenty that link against CUDA nowadays.

[roflmaostc]

Any progress @PaulVirally?

I think if that would work, people would immediately try out and play with it (including me)!