Enable configuration reuse

[starkos]

What problem will this solve? Allow project authors to script reusable blocks of configuration, which can then be used for:

• Providing all necessary configuration to consumers of a library or project. Just linking a library is often not enough: preprocessor symbols, include directories, and additional linker dependencies may also be required. This feature would allow project authors to specify all of this configuration, including the links() call, in a block that can be reused by consumers.

• Specifying common configuration once, and reusing it across multiple projects and configurations. For instance, setting a standard for how a debug build should be configured, or the listing the settings necessary to use a third-party library.

We've also called this feature "usages", and have talked about them quite a lot (see "What other alternatives…" below).

What might be a solution? After going through all of the discussions and my own notes over the years, the example below is how I think it should be implemented. Feel free to disagree with me in the comments; I'll update this write-up to summarize the discussion.

This is a contrived example to show the main points. Please use your imagination to scale it up to meaningful project sizes, with all the filters for platforms, etc. that you would normally need to script. All names are my proposals only and are open for discussion.

-- Projects are defined as before. Any configuration provided here is
-- "internal", and only used while building the project. To pull in
-- shared configuration, use the `uses` statement.
project "MyLibrary"
    kind "StaticLib"
    files { "**.h", "**.cpp" }
    uses { "MyLibrary:common" }

-- This shared block contains configuration that is used both to build
-- the library, and for consumers of the library. The name is arbitrary 
-- and assigned by the script author. Special characters are okay in 
-- block names.
block "MyLibrary:common"
    filter { "configurations:Debug" }
        defines { "MYLIBRARY_DEBUG" }
    filter { "configurations:Release" }
        defines { "MYLIBRARY_RELEASE" }

-- This shared block contains the configuration required only by 
-- consumers of the library, and not the library project itself.
-- It is okay to use the same name as the project.
block "MyLibrary"
    uses { "MyLibrary:common" }
    defines { "MYLIBRARY_API" }
    links { "MyLibrary", "Dependency1", "Dependency2" }

-- A project that wants to consume the library can pull in the shared
-- "public" configuration block.
project "MyExecutable"
    uses { "MyLibrary" }

Some usage points that might not be obvious, or would need to be considered during development:

• A block only gets "used" once. If it appears again in a uses() list it will be ignored. For example, if you try to "use" two libraries which share a common dependency.

• To maximize reuse, Premake should be smart enough not to link a project against itself. That way a project can "use" its own public configuration without creating a circular dependency.

• If no shared block exists, uses "MyLibrary" simply links the project, i.e. it becomes links "MyLibrary".

What other alternatives have you already considered? As mentioned, this has been discussed before. These are the most recent and relevant references:

• PR #956 presents an alternative approach, and sparked quite a lengthy conversation of pros and cons. This comment summarizes the discussion points.

• Issue #1215 expresses some other pain points that this feature potentially help solve.

• moomalade/premake-usage and Meoo/premake-export are other attempts at an implementation.

Some advantages of the approach that I proposed here:

• Works for any type of configuration reuse; not tied to any particular use case
• Helps minimize the number of filter calls required
• Fits into the existing container+filters approach that we're using for everything else
• Doesn't require adding any special logic to support "building" vs. "using"

Feedback appreciated! Feedback is welcome and appreciated in the comments. I'll keep this updated to summarize any key points or alternative approaches that get raised. When expressing a preference, it would be helpful if you could provide some explanation. And if you don't like these names, please try to provide some alternatives you do like.

(You can now support Premake on our OpenCollective. Your contributions help us get features like this shipped!)

[evil-jester]

Just wanted to point out another module that tries to address this issue: https://github.com/Meoo/premake-export

[ratzlaff]

I believe this would reduce some duplication in the scripts I maintain here at work. Looks good sofar, interested to see it in practice.

[noresources]

+1

I believe this would reduce some duplication in the scripts I maintain here at work. Looks good sofar, interested to see it in practice.

Same thoughts for me. This is the key feature I am waiting to rewrite and improve the build scripts I use at work.

[samsinsane]

@starkos Somewhat off topic, this seems like it would replace the tags API, would you agree with that? It seems to do a similar job, but also a better job if you factor in the consistency across other use cases.

[starkos]

this seems like it would replace the tags API

TBH, I've never used the feature, and the documentation is…lacking. But going from memory, I think they might be complementary. You could still define and filter on your own custom tags, this would give you another way to do it.

[samsinsane]

That's fair enough, it didn't really gain much traction probably due to the documentation. Another thing that I didn't really think about is that uses { "something" } would likely require a block "something" but you can have a billion tags and use none of them.

[ClxS]

I'm not sure how much use we'd get out of the block config inheritance as workarounds exist. Our approach to this at work has to just been to either: 1) Override "project" to inject our default definition 2) Add new "project" words, i.e. gtest "ProfTest", which sets up the project and all default configuration specific to gtest projects 3) Wrap it all in a function and call it on the first line after defining the project

We have also implemented something similar to the original PR (#956) which investigated a "using" field: We've made a module which will generate pubx.. definitions, where x can be either links, defines, includedirs, postbuildcommands, etc. We also added a "references" property which is essentially a non-linking link. It allows for inheritance without bloating intermediate static libs. Any library which links against something with a pubx field will inherit that same pubx field. This allows us to do something like:

project "CUDA"
   publibdirs { path.getabsolute('lib') }
   filter { "platforms:Windows" }
      publinks { 'cuda.lib' }
   filter { "platforms:Linux" }
      publinks { 'cuda.a' }
   filter {}

project "SomeRenderLib"
   references { "CUDA" }

project "X"
   links { "SomeRenderLib" }

In this example SomeRenderLib will inherit the publibdirs and publinks from the CUDA project, which in turn the project "X" will inherit as well as performing a proper link against SomeRenderLib. The inheritance is applied just prior to postBake so there's no order dependency issues.

It's worked very well for us, and we use it to hold together a 200+ project, 5+ platform modular game engine.

[smbradley]

I think we could map our packagemanager solution to this approach. One thing worth noting is that we get a lot of use out of include/link dependency separation, which would lead to something akin to:

project "MyLibrary"
    kind "StaticLib"
    files { "**.h", "**.cpp" }
    uses { "MyLibrary:common" }

block "MyLibrary:common"
    includedirs { "include" }

block "MyLibrary"
    uses { "MyLibrary:common" }
    links { "MyLibrary" }

project "MyOtherLibrary"
    kind "StaticLib"
    uses { "MyLibrary:common" }

project "MyExecutable"
    uses { "MyLibrary" }

Which could get cumbersome across a large project unless people standardized on naming conventions. For us we'd probably wrap the block creation and uses to ensure this.

That's fair enough, it didn't really gain much traction probably due to the documentation. Another thing that I didn't really think about is that uses { "something" } would likely require a block "something" but you can have a billion tags and use none of them.

We've found them useful to group projects and/or configurations into common categories and then filter on those conditions or to search for a subset of projects etc. Eg. tagging a set of configurations as internal which then influences build settings in various projects.

[simonrenger]

Isn't this already solved with https://github.com/premake/premake-core/wiki/Usages proposal https://github.com/moomalade/premake-usage

[starkos]

This is discussed in the OP.

[learn-more]

Are you already set on the block name? I would argue that interface would communicate better what the purpose is.