Fails to precompile on CI

[BeastyBlacksmith]

I got a project depending on ViennaRNA.jl which fails to precompile (but only in CI jobs) with the following error

┌ Error: /root/.julia/artifacts/725e603448ac6c44bfbbf2df16f3225673d1ff38/include/ViennaRNA/utils/structures.h:140:10: fatal error: 'stdio.h' file not found
└ @ CBinding ~/.julia/packages/CBinding/kBUap/src/context.jl:377
ERROR: LoadError: Errors parsing C code
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] parse!(ctx::CBinding.Context{:c})
    @ CBinding ~/.julia/packages/CBinding/kBUap/src/context.jl:388
  [3] clang_str(mod::Module, loc::LineNumberNode, lang::Symbol, str::String, opts::String)
    @ CBinding ~/.julia/packages/CBinding/kBUap/src/context.jl:481
  [4] var"@c_str"(__source__::LineNumberNode, __module__::Module, exprs::Vararg{Any})
    @ CBinding ~/.julia/packages/CBinding/kBUap/src/context_c.jl:5
  [5] include
    @ ./Base.jl:418 [inlined]
  [6] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1318
  [7] top-level scope
    @ none:1
  [8] eval
    @ ./boot.jl:373 [inlined]
  [9] eval(x::Expr)
    @ Base.MainInclude ./client.jl:453
 [10] top-level scope
    @ none:1
in expression starting at /root/.julia/packages/ViennaRNA/7mJxT/src/ViennaRNA.jl:21
in expression starting at /root/.julia/packages/ViennaRNA/7mJxT/src/ViennaRNA.jl:1

So it seems this is not really selfcontained and the container is missing some C dependencies, if I understand correctly

[marcom]

Hi, thanks for the bug report! I have encountered the same issue when trying out the package on windows. I'm using CBinding.jl to interface with the C code. It works by parsing the C code at runtime using Clang_jll, and looks like it errored out on your machine because stdio.h can't be found.

I'll try to find a quick fix, the best long-term solution would be to generate the bindings only once and not on every run.

[marcom]

A quick fix for you might be to install a package like libc-dev or musl-dev (that has the standard C header files such as stdio.h) in the CI container.

[BeastyBlacksmith]

Hmm, I installed linux-libc-dev on a ubuntu image, but it didn't help. But adding build-essential did

[marcom]

Sorry for the slow responses! Glad you could fix your problem.

I have thought a bit more about the problem, this is my understanding at the moment:

The reason i used CBinding was so i could also access and manipulate ViennaRNA C structs (possibly deeply nested), for example the energy parameters. In C, the struct layout (due to padding etc) can be different depending on target ABI, which means the safest way for CBinding to access struct fields is to use a C compiler on the target machine when loading the package. This has the downside of increased overhead on package loading (though it seems ok for this package) and the end-user needing all the necessary C system headers (because Clang_jll doesn't supply them).

In case one doesn't want to access struct fields, this generation of bindings at runtime isn't necessary and one could statically generate all C bindings once and distribute them with ViennaRNA.jl, the way most C bindings in Julia are done.

Maybe it would be possible to make two packages, one a high-level interface that doesn't need CBinding and a low-level interface that does.

[BeastyBlacksmith]

Alternatively you could use Requires.jl to only require CBinding when needed.

[BeastyBlacksmith]

This just bit me again on a windows server without a c compiler and installing one behind a proxy is currently failing.

[marcom]

I'm not sure how to reliably check for the presence of C standard headers with Requires.jl, so i'm going to create another package ViennaRNA_CBinding.jl that uses CBinding for low-level access.

This package will then become the high-level interface and will be self-contained.

Does this sound reasonable to you or am i missing some trick that would make it all work with Requires.jl?

[marcom]

I think i've got the high-level interface working now on the main branch, it's now pre-generated with Clang.jl on the developer's machine (not at run-time).

I'm still improving some things and then i'll release a new version.

If you want to try it you can do so via ] add ViennaRNA#a5d70b3 (the most recent commit) or ] add ViennaRNA#main.

I just tested it under Windows and everything seems to work, so no C standard header files needed anymore.

[marcom]

This should work now with ViennaRNA.jl v0.3.0. Please reopen the issue if it doesn't work for you. Thanks!