sort out design for foreign code in Haskell libraries

[bos]

(Imported from Trac #704, reported by @dcoutts on 2010-06-21)

Currently we often happily add little bits of C glue code into Haskell packages. This is perfectly reasonably but we also want to be able to do things like linking multiple versions of a Haskell package into a single application. That does not work if the C symbols are visible outside of the package (see #701).

It seems reasonable to require that C code in a Haskell package respects the properties that we want for Haskell packages. However we must also allow for libraries that are foreign code and not Haskell libraries at all.

For example, the zlib binding package ships a complete copy of the zlib C code for platforms that do not have it natively. Unlike bits of FFI glue code, it is unreasonable to expect to change the zlib code to conform to Haskell library requirements.

Perhaps what we need is to explicitly identify foreign libraries and hold them to different (lower) standards compared to Haskell libraries. One idea would be to draw together the features proposed in these tickets:

• #276 : support for convenience libraries
• #148 : producing foreign libraires

The idea with convenience libraries is that a package can have multiple libraries in it, even though at most one can be the library that has exposed modules. Other libraries could be used by executables or the main library in that package. The main purpose is sharing but it could also be a way to separate the requirements of haskell code and foreign code. Producing foreign libraries is about making libraries (usually but not necessarily containing Haskell code) that export a foreign C API and are not intended to be used by other Haskell code. Combining these two features would allow for the zlib example to work ok. We would declare a foreign library with the zlib C code and make the main Haskell binding library depend on it. Note that packages that depend on the binding package would not directly link to the C library (this is significant for shared libs). The key constraint for foreign libraries is that we cannot link more than one version into a single application (because the symbols would clash), where as we may assume that it is perfectly safe to link multiple versions of a Haskell package. Any design must also consider symbol visibility. This is relevant both for shared and static libs. With static libs it is hard to avoid every symbol being globally visible. With shared libs there is greater control over symbol visibility but also the restriction on some systems that each shared lib must resolve all symbol references. This means we must more explicitly identify which libraries import symbols from others.

[bos]

(Imported comment by @dcoutts on 2010-06-21)

See also #757.

[BurningWitness]

I don't know if this is a correct topic to link to, but it seems close enough to mine.

I have written quite a few FFI bindings. I figured out everything there is on the Haskell side of things, however my understanding of how to set the project up in a portable manner is incredibly foggy. Up until now I've resorted to picking a version of the C library, copying the entire thing into a repository and setting up around it (e.g. freetype2.cabal), however a few days ago I was rewriting botan-bindings.cabal which reminded me that libraries can also be shared.

Ideally the C library setup is something the user should do themselves, choosing whether to compile static or dynamic, specifying directories and picking macros (also building static libraries, e.g. invoking make or configure, but this will have nothing to do with Cabal files). Based on the reference this is not currently possible, the only thing the user can do is add extra FFI directories.

I don't know if I'm missing documentation for certain functionality or if this is a longstanding issue that has not been addressed due to how few people touch the FFI.

[Mikolaj]

Thank you for your kind interest. If any support is provided other than what the manual says, there would be evidence of it on Hackage (I wouldn't know, I rarely look inside such packages, e.g., into their .cabal files). I guess some people could call make, configure or gcc from custom Setup.hs scripts, but it's a mechanism we are trying to move away from (we may offer hooks instead or perhaps https://github.com/haskell/cabal/pull/8969/files handle some use cases. In any case, in the absence of a ready toolbox, it still makes sense to outline concrete orthogonal needs based on simplified use cases as tickets here and then RFC them on discourse.haskell.org, perhaps pinging maintainers of the more notorious packages that could benefit.