CBinding not finding DWDataReaderLib64 library

[Svampbob1001]

I could not get the package to compile. CBinding is not able to find the DWDataReaderLib64.dll file in the package path "C:\Users\JGELWX.julia\packages\DWDataReader\qLQOB\src.." I can see that DWDataReaderLib64.dll is located there but I don't know what the problem is or how to fix it.

This was tested using Julia 1.8.5

Failed to precompile DWDataReader [d50a3a05-91c6-42d5-aeb6-a7244554f0d0] to C:\Users\JGELWX\.julia\compiled\v1.8\DWDataReader\jl_4861.tmp. ERROR: LoadError: Failed to find library DWDataReaderLib64 in C:\Users\JGELWX\.julia\packages\DWDataReader\qLQOB\src\.. Stacktrace: [1] error(s::String) @ Base .\error.jl:35 [2] configure!(ctx::CBinding.Context{:c}) @ CBinding C:\Users\JGELWX\.julia\packages\CBinding\PrYH5\src\context.jl:287 [3] CBinding.Context{:c}(::Module, ::String, ::Vararg{String}) @ CBinding C:\Users\JGELWX\.julia\packages\CBinding\PrYH5\src\CBinding.jl:149 [4] top-level scope @ C:\Users\JGELWX\.julia\packages\CBinding\PrYH5\src\context.jl:436 [5] include(mod::Module, _path::String) @ Base .\Base.jl:419 [6] include(x::String) @ DWDataReader C:\Users\JGELWX\.julia\packages\DWDataReader\qLQOB\src\DWDataReader.jl:1 [7] top-level scope @ C:\Users\JGELWX\.julia\packages\DWDataReader\qLQOB\src\DWDataReader.jl:3 [8] include @ .\Base.jl:419 [inlined] [9] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing) @ Base .\loading.jl:1554 [10] top-level scope @ stdin:1 in expression starting at C:\Users\JGELWX\.julia\packages\DWDataReader\qLQOB\src\file.jl:9 in expression starting at C:\Users\JGELWX\.julia\packages\DWDataReader\qLQOB\src\DWDataReader.jl:1 in expression starting at stdin:1

[fleimgruber]

Thanks for reporting, I will test with Julia 1.8.5 and get back to you.

[fleimgruber]

@Svampbob1001 please try with https://github.com/fleimgruber/DWDataReader.jl/commit/b4275b546e0250624d511b383708f2d1dcca15d0. I also corrected the relevant parts in the README that were promising too much testing (copy-paste artifact).

[fleimgruber]

My last comment was about using the development version from git. If you want to wait for the new version to reach the Julia registry, please give me some more time so that I can understand and follow the required steps in https://github.com/JuliaRegistries/Registrator.jl/blob/master/README.md.

[Svampbob1001]

Thanks for your quick respons 😃 Still not working. I ran the commands marked in bold below. You can also se the other packages I am running in the last command. I don't know if it has anything to do with the windows Environment I am using. I don't know if it has anything to do with the compiler as stated in the README. I am not familiar with CBinding and c/c++ compilers.

On Windows, there is no support for MSVC, only for MSYS2 MinGW gcc.

I have the package python package https://pypi.org/project/dwdatareader/ working, but not with Julia PyCall.

(@v1.8) pkg> add https://github.com/fleimgruber/DWDataReader.jl.git Updating git-repo https://github.com/fleimgruber/DWDataReader.jl.git Resolving package versions... Updating C:\Users\JGELWX\.julia\environments\v1.8\Project.toml [d50a3a05] + DWDataReader v0.1.0 https://github.com/fleimgruber/DWDataReader.jl.git#master Updating C:\Users\JGELWX\.julia\environments\v1.8\Manifest.toml [d50a3a05] + DWDataReader v0.1.0 https://github.com/fleimgruber/DWDataReader.jl.git#master

(@v1.8) pkg> precompile Precompiling project... ✗ DWDataReader 0 dependencies successfully precompiled in 5 seconds. 408 already precompiled.

ERROR: The following 1 direct dependency failed to precompile:

DWDataReader [d50a3a05-91c6-42d5-aeb6-a7244554f0d0]

Failed to precompile DWDataReader [d50a3a05-91c6-42d5-aeb6-a7244554f0d0] to C:\Users\JGELWX.julia\compiled\v1.8\DWDataReader\jl_F20C.tmp. ERROR: LoadError: Failed to find library DWDataReaderLib64 in C:\Users\JGELWX.julia\packages\DWDataReader\Km9LQ\src.. Stacktrace: [1] error(s::String) @ Base .\error.jl:35 [2] configure!(ctx::CBinding.Context{:c}) @ CBinding C:\Users\JGELWX.julia\packages\CBinding\PrYH5\src\context.jl:287 [3] CBinding.Context{:c}(::Module, ::String, ::Vararg{String}) @ CBinding C:\Users\JGELWX.julia\packages\CBinding\PrYH5\src\CBinding.jl:149 [4] top-level scope @ C:\Users\JGELWX.julia\packages\CBinding\PrYH5\src\context.jl:436 [5] include(mod::Module, _path::String) @ Base .\Base.jl:419 [6] include(x::String) @ DWDataReader C:\Users\JGELWX.julia\packages\DWDataReader\Km9LQ\src\DWDataReader.jl:1 [7] top-level scope @ C:\Users\JGELWX.julia\packages\DWDataReader\Km9LQ\src\DWDataReader.jl:3 [8] include @ .\Base.jl:419 [inlined] [9] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing) @ Base .\loading.jl:1554 [10] top-level scope @ stdin:1 in expression starting at C:\Users\JGELWX.julia\packages\DWDataReader\Km9LQ\src\file.jl:9 in expression starting at C:\Users\JGELWX.julia\packages\DWDataReader\Km9LQ\src\DWDataReader.jl:1 in expression starting at stdin:1

(@v1.8) pkg> st Status C:\Users\JGELWX\.julia\environments\v1.8\Project.toml [d43a6710] CBinding v1.0.10 [336ed68f] CSV v0.10.9 [13f3f980] CairoMakie v0.10.4 [5ae59095] Colors v0.12.10 [a6e380b2] ControlSystems v1.7.2 [5a033b19] CurveFit v0.5.0 [d50a3a05] DWDataReader v0.1.0 https://github.com/fleimgruber/DWDataReader.jl.git#master [a93c6f00] DataFrames v1.5.0 [0c46a032] DifferentialEquations v7.7.0 [f6369f11] ForwardDiff v0.10.35 [e9467ef8] GLMakie v0.8.4 [a98d9a8b] Interpolations v0.14.7 [b964fa9f] LaTeXStrings v1.3.0 [23fbe1c1] Latexify v0.15.18 [23992714] MAT v0.10.4 [961ee093] ModelingToolkit v8.52.0 [eadc2687] Pandas v1.6.1 [91a5bcdd] Plots v1.38.9 [c3e4b0f8] Pluto v0.19.24 [438e738f] PyCall v1.95.1 [1fd47b50] QuadGK v2.8.2 [b0e4dd01] RollingFunctions v0.7.0 [3f865c0f] ScatteredInterpolation v0.3.6 [0c5d862f] Symbolics v5.2.0 [bd369af6] Tables v1.10.1 [9e3dc215] TimeSeries v0.23.1 [f269a46b] TimeZones v1.9.1 [1986cc42] Unitful v1.13.1 [fdbf4ff8] XLSX v0.9.0

[fleimgruber]

I am guessing this has to do with your Windows environment, i.e. a supported compiler is missing. The relevant check is made in the C preprocessor stage here. Please see also for background: https://github.com/analytech-solutions/CBinding.jl/issues/109.

@krrutkow is there a way to debug the compilations that CBinding.jl triggers here to see the output and confirm that the reason is not having MSYS2 MinGW gcc?

[fleimgruber]

@Svampbob1001 If you feel adventurous, you can try installing https://www.msys2.org/ and the MinGW C compiler toolchain. I find the MSYS2 environment nice to have on Windows.

[Svampbob1001]

I did try to install MSYS2 and installed the following tool chain: ucrt64/mingw-w64-ucrt-x86_64-gcc 12.2.0-10 (mingw-w64-ucrt-x86_64-toolchain) GNU Compiler Collection (C,C++,OpenMP) for MinGW-w64 But no luck. I still saw the exact same error. It fails specifically on line 9 in file.jl, which is before the check in the C preprocessor stage at lines 11-15.

[krrutkow]

Sorry, I'm not familiar with the Windows setup being used here, but CBinding would be using something like Libdl.find_library("DWDataReaderLib64", "C:/Users/JGELWX/.julia/packages/DWDataReader/qLQOB/src/..") to find the library, so perhaps try some variations of that command to find the library. Also, you should try loading the library with something like Libdl.dlopen("C:/Users/JGELWX/.julia/packages/DWDataReader/qLQOB/src/../DWDataReaderLib64") to see if it is possible that the library doesn't load successfully at all in your environment. The find_library documentation says it only finds libraries that "can successfully be dlopen'd".

[fleimgruber]

@Svampbob1001 sorry for the delay. I can reproduce this on a fresh Windows system, will look into it.

[fleimgruber]

@Svampbob1001 Some update: Thanks to the debug instructions by @krrutkow I found the issue and a workaround, see also https://discourse.julialang.org/t/private-package-dll-gives-access-is-denied/58295/3. The issue is that Windows does not like the default file permissions for Julia-installed packages when opening DLL files. This is a temporary workaround:

Julia> lib = "C:/Users/user/.julia/packages/DWDataReader/AfhYx/DWDataReaderLib64.dll"
Julia> chmod(lib, filemode(lib) | 0o755)
Julia> Libdl.dlopen(lib)
Ptr{Nothing} @0x00007fffaae60000

I will ask the Julia community what the recommended approach would be here so that we can properly handle this on the library code level - or maybe @krrutkow already has an idea?

[fleimgruber]

@Svampbob1001 It would be great if you could test this again with 4a91205bee5f85741dca858cd1e219ba5587ae67, i.e. please try again to install from git as you did in https://github.com/fleimgruber/DWDataReader.jl/issues/3#issuecomment-1512755032. You should see it working (or the next error, possibly due to basetsd.h, but one step at a time...).

[fleimgruber]

@Svampbob1001 I noted in https://github.com/fleimgruber/DWDataReader.jl/issues/3#issuecomment-1518579859 that you used the UCRT toolchain. I would suggest you use the x64_86 toolchain (from MSYS2 MINGW64 shell):

pacman -S --needed base-devel mingw-w64-x86_64-toolchain

and also add these two paths to the $PATH environment variable (adapt for your MSYS2 installation directory):

• C:\Users\user\opt\msys64\usr\bin
• C:\Users\user\opt\msys64\mingw64\bin