Statically compiling libraries and Lua modules

[davidm]

STATIC/SHARED/MODULE in add_library override BUILD_SHARED_LIBS, which in turn seems to default to false. Therefore I think it may be a bad idea to hardcode add_library(... SHARED ...) in CMakeLists.txt files in general. We may want to instead use some wrapper function in dist.cmake that will select either shared or static based on a global user preference. Sometimes both are even wanted, which sometimes complicates things.

Also note that some toolchains (e.g. sdcc -- sdcc.sourceforge.net) don't support building shared libraries, so it will force STATIC regardless, as this cmake warning shows: "ADD_LIBRARY for library foo is used with the SHARED option, but the target platform supports only STATIC libraries. Building it STATIC instead. This may lead to problems."

The above concerns C libraries in general. Statically linking modules for any language we support (e.g. Lua) is a separate question. Probably we would want some mechanism to package.preload them in linit.c (note: the 5.2 version handles this more correctly). I'm thinking the compilation order would be this: (1) build a vanilla interpreter without modules, (2) build static libraries of modules, and (3) build another interpreter using #2. Step #3 can be repeated any number of times (as different variants) for various combinations of #2, and you may iterate back-and-forth between #2 and #3. Related possibilities include compiling source modules to C libraries via bin2c for linking in #3 as a single file, like various bundling utilities have done, so this overlaps those efforts. Anyway, I don't see obstacles here provided each static library publishes the package names of its modules and the symbol names of its registration functions in some standard way (actually you can probably infer the latter from the former, and the former are already in the CMakeLists.txt).

[drahosp]

I started a new LuaDist/lua@new_dist_structure with some significant modification to dist.cmake. It now relies on BUILD_SHARED_LIBS but defines sane defaults including RPATH linking. dist.info is now used in configuration and package generation. Additionally I removed support for the SKIP_ defines and decided to use COMPONENT properties instead. I will add the lua.cmake and FindLua.cmake files there later. It would be probably best to use that branch in relation to issue #49.

The use case when static and shared libraries are defined it is not directly supported in CMake and I do not want to complicate things too much. So I prefer to look into this later, hopefully CMake will support it to make it easier.

Statically linking lua modules would be great and your suggested approaches sound good. I don't see a problem to add support to the new lua.cmake macros for other use cases such as this. However for now I'm focusing on dist.cmake and batteries release. After that is off the table I want to focus on Lua 5.2, tests and travis with CDash etc. Other use cases such static compilation and perhaps bundling whole Lua based applications is exactly what I would like to add after that.

I leave this open as long as there are modules with hardcoded SHARED libraries.

[davidm]

Looks ok. I pushed some minor changes.

I think tests (with make test without CDash or anything fancy) could be enabled sooner rather than later though. It doesn't look like travis-ci is testing even though it has a -test=true. Most modules I think will test fine or can be easily made to do so, assuming of course it's compiling ok (if not, its something that will need to be fixed in the batteries release). Recent changes to lua.cmake address the main problem with testing, and I've fixed up a few modules.

[drahosp]

Yes, CDash is a bonus. For now I made a simple Wiki Status page that I abuse as sort-of CDash.

As for tests ... they seem to be disabled in the luadist cli, I make sure it works as expected before switching bootstrap to luadist-git which works better.

[drahosp]

I tried the approach @davidm mentioned and modified the lua.cmake macro to support a LUA_STATIC_MODULE option. With this option on, lua modules will be compiled statically and placed in the same location as dynamic modules. However I have also modified Lua (branch static) to search for these modules and link them in. The process automatically generates all luaopen_x functions in linit.c using the location of the static modules. Same approach will work for lua 5.2. I will merge this in once I test it on all platforms and update luajit and lua-5.2.

[pforai]

Hi @drahosp What is the status of this, and are there any docs that mention how to do that? I would really need this!

[drahosp]

This feature requires some additional infrastructure, while I have implemented almost everything that is needed for this to go live. I still need to make a tool that uses 'luadist-git' to generate CMake builds that use 'external_project_add'. This would basically generate a CMake super project that would link every modules statically. It is possible to link binded libraries in also but that is limited by their respective licenses.

Sorry, there is currently no documentation on this feature as it still requires significant work in order to make it usable with simple instructions.

[pforai]

Thanks for the detailed info and keep up the great work!

Is the static branch of the lua5.1 repo in a usable state? Ie does it build a lua.a and static lua executables?

[drahosp]

No it is not, it is an old WIP branch that I used to transfer changes to be tested on multiple machines/architectures. I will remove it so it does not cause further confusion.

[mikegarts]

The automated static build feature sounds great. It can save a lot of time when one tries to create a static build of lua+some set of packages. Especially when this involves some cross compilation

[acki-m]

@drahosp What is the current status here? How can I import lua as external project an link as static library via CMake? I did not found any LUA_STATIC_MODULE option in cmake.

[drahosp]

Sadly due to my time constraints, this feature never managed to end up in the project. There is no development on the LuaDist project for more than 3 years now. Sorry. EDIT: Replied from wrong account earlier.