'links' doesn't follow dependency on static libs projects

[dsvi]

project "A"
  kind "StaticLib"
...
project "B"
  kind "StaticLib"
  links {"A"}
...
project "C"
  kind "StaticLib"
  links {"B"}

project "App"
  kind "ConsoleApp
  links {"C"} --> not going to be built, since B and A are not included

When there is links dependency on another project, which is a static lib, premake should include all the static lib dependencies of that as well. Otherwise user has to include all of them explicitly in the resulting app or DLL. ( links {"C", "A", "B"} for the App in the example above) Which is troublesome, since the list of libs might be conditional (platform dependent, for example) and user will have to copy/paste the conditions several times over. Note, this isn't a problem for dlls, only for static libraries. Since dll has all it's dependencies built in already.

[mikisch81]

Also seen this issue, it is specific in gmake, in Visual Studio this problem does not happen.

[dsvi]

The reason why it does not happen in VS, is because VS does it itself. From quick look at the sources of premake the function config.getlinks(cfg, kind, part, linkage) simply doesn't follow any dependencies for a static lib In general premake lacks the notion of dependencies at all. You can't tell it "to use this library add those defines, include paths, and libs" like you can in more modern build systems.

[starkos]

We tried enabling this a while ago, and then rolled it back. I don't remember the exact reasons; if I get a chance I will try to locate the related PRs or discussion threads if I can.

So heads up, there are reasons not to enable this. I ended up implementing a simple override for the one place I needed it to work, I'll try to dig that up too.

[tvandijck]

Oh yeah, we tried this, and made a commit for that... which got quickly rolled back because premake times for some of @starkos his projects went from seconds to minutes. I'm sure we did something wrong...

Arguably however a static lib linking against another static lib creates a hard order dependency on build order and a sync point.... which in a paralllel build isn't really very efficient....

For example, in the example provided above... project B can't start compiling until project A is done compiling. Although technically not entirely true, since all projects could easily compile, they just have to wait linking... but that is just not how Visual Studio treats these dependencies... so in practice what you'll see is that B waits for A to finish then C waits for B to finish, and then App compiles and links... For a small project not really a big deal... but now throw a 5 million line codebase at it, and try building it through XGE (Incredibuild), and the entire project serializes...

So our solution at the time (what we tried to merge, but failed at), was to treat "links" in libs are "virtual dependencies" in premake, and resolve them down to the application. So despite you writing what you wrote above, premake would treat it as if you wrote

project "App"
  kind "ConsoleApp
  links {"A", "B", "C"}

because from a parallel build perspective that is most efficient... A, B, and C would all be order independent, and only App would have to wait for all three to finish...

Anyway, ultimately I think with whatever solution we end up with, this is a very important consideration. I'd be happy to make another attempt some day... but it's non-trivial to get right and fast...

[samsinsane]

While I like the idea of what @tvandijck has suggested, it kind of only solves part of the problem. Should we look at creating a proper API instead?

[tvandijck]

Well, what kind of API would that be though??

"links" already kind of is that API, whether we setup the project to actually "link" or we setup the project to achieve the same final result... I think is an implementation detail of premake.

For example there is this paper that talks about a similar 'issue' with makefiles. I think someone else recently brought this up as well... http://aegis.sourceforge.net/auug97.pdf It basically talks about the inefficiencies of considering "libraries" single units.. because ultimately if you simply look at the content of a lib, it's just a bunch of obj files and a TOC.

Ultimately it is the application linker that takes all those obj files together and creates the final binary. Meaning that we could compiling all .obj files completely independently if we didn't have this concept of libraries as units.

So treating a "links" in a staticlib as a completely virtual concept makes technically no difference to the final result. The "dependson" keyword already allows us to control build order dependency. So if build order is important due to code generation, or something else, then the dependson should already be used for that instead of the "links"..

I think this could be done with the "links" keyword fairly easy, but it's a breaking change due to it having an effect on build order, which in some cases was the desired effect rather then actually linking.

That aside however.... while this could be fixed easily with "links" to propagate libraries, I'd want the same for "includedirs"... it's super annoying that if I use 'zlib' in a public header file of my library that I now need to "includedirs { '../../zlib/include' }" in every project that uses my library...

And so we arrive at the "uses" API we've dancing around for the last year orso... So you are right.. we could create a proper API to capture ALL of these things, including things like "hey, if you use me, then you need this include folder, these set of defines, etc"... That however would be in addition to 'fixing' the links api in my opinion.

[samsinsane]

That however would be in addition to 'fixing' the links api in my opinion.

You're right, I didn't quite think through the dependency API properly. What I was thinking would result in something like this:

-- lib.lua
  links { "zlib" }
  dependencies {
    links = { "zlib" },
  }

-- app.lua
  links { "lib" }

Obviously, specifying "zlib" twice is silly, and what you were suggesting avoids that. I'd still like this issue to remain open until we have a dependencies API, rather than just fixing the links API. As you said, specifying the include dirs and what not is a bit annoying.

[tvandijck]

FWIW: https://github.com/Meoo/premake-export

kind of makes an attempt at doing the above..

Furthermore, I don't think you would specify links {"zlib"} for lib.lua, since that doesn't really have any purpose... the dependencies you setup later already serves the purpose that anyone using lib.lua will get a links on 'zlib'..

Agreed though, let keep this one open for further discussion...

[dsvi]

For dependencies i am just using separate exports.lua with

includedirs {
  ".",  -- to my big surprise this becames "libs/libA" in project "App" below
}
libdirs {
  "bin/%{cfg.buildcfg}",
}
links { "libA" }

which i then include at the project, which uses it:

project "App"
  kind "ConsoleApp"
  ...
  include "libs/libA/exports.lua" 

See no reasons why exports.lua can't include another exports.lua from some other lib in cascade

Generally i don't hold all the stuff in premake5.lua, but have separate files lib.lua, which contains only the lib project, tests.lua - the lib tests project, and premake5.lua, which contains only workspace and includes the lib.lua and tests.lua at the bottom. Also the exports.lua which i include at the projects, where the lib is used, together with lib.lua, which is included at the workspace level of the user project. This allows me to easily include the libs, where i need them. Kind of emulates "dependency" functionality

workspace "App"
...
include "libs/libA/lib.lua"

project "App"
  kind "ConsoleApp"
  ...
  include "libs/libA/exports.lua" 

see example

Sly, aren't i?

[dsvi]

ehm... cascade includes actually don't work. i can't include same exports.lua twice dofile instad of include works fine though

[pvorisek25]

Is some work done on this topic in 2020?

[dsvi]

People just use CMake nowadays.

[detly]

CMake has explicitly ruled this out, so don't look for it there.

[dsvi]

CMake does have a way to include a library with all its dependencies included automatically. Premake doesn't. And generally CMake is head and shoulders above Premake. The popularity of both tools reflects their usefulness.

[samsinsane]

@dsvi it sounds like you've found the right tool for your task. Thank you for sharing your opinion on CMake and Premake with us. However, I have to ask you to refrain from posting antagonistic comments otherwise we'll be forced to block you from the project.

[lufsc]

I still think that premake should strive to have the same behaviour on all build systems. As visual studio already links the dependencies, the gmake or gmake2 targets should do the same. As far as I can see, visual studio does not copy the include directories, so all that's needed is linking the libraries, that the library depends on.

PS: Sorry for the duplicate issue.

[JumpyLionnn]

Are there any updates?