Premake suitability for more complex projects

[dumblob]

I didn't follow Premake development for some time and today stumbled upon the following comment on reddit:

Just like all of these half-arsed make replacements, Premake works very nicely for the limited set of use-cases that the author knew about when he started the project. It then fails to extend nicely into all of the corner cases that you find you need when you have a real project to build. Source: I have used Premake a lot.

Can you elaborate on that? I've been using Premake for a few months now and I haven't run into any such issues.

Sure. The first problem is scale - nothing in Premake is built to cope with large projects. Just running Premake on a build with hundreds of "projects" can take tens of minutes, even hours. That's all down to weak choice of O(N) or even O(N2) algorithms in the Lua code. It's simple to fix with a bit of refactoring, but it's a pretty clear signal about the maturity of the code. The generated Makefiles are far from efficient too. An up-to-date project can still take many minutes to say "nothing to do". Secondly, Premake has no way of properly incorporating file-specific build steps. Let's say you need to use some extra optimisations for certain files, or if ICC produces better object code in certain circumstances - bad luck. Thirdly, modern C++ builds can be much more complicated that the simple compile-link cycle that Premake implements. If you want to incorporate LTO or PGO into your build, Premake has nothing for you. I could go on, but I think that's enough to give you the idea. The big problem is that none of these limitations are apparent when you start out. Nobody wants to go to the trouble of using LTO - it's only towards the end of your schedule, when you realise that your executables are 50% too slow, and you've exhausted all of the simple optimisations that you start to wonder whether LTO might help you. Same for all of the other things. Don't get me wrong. I used Premake, and extensively modified it to overcome many of these limitations. There is a solid idea in there, but it simply lacks the maturity of older tools. That's inevitable of course - it's still in its infancy. You can see the same pattern in most "new build system" tools. The initial authors only see their own limited use-case, and can't fathom why traditional tools need to be so overwrought, to solve such a "simple" problem. They craft a new tool that works for them, and is much simpler. When other people start jumping aboard and asking for ways to address their own use-cases the tool starts to sprout more and more special-case features. Eventually, either it dies, or you end up with something that's every bit as complicated at Autoconf.

I've found just the LTO issue solved. My question is how about the others?

[tvandijck]

Well, I don't really know how to respond to that ;) I'm sure like many other tools Premake has its pros and cons. And ultimately it is up to you to decide what is the best tool for the best job...

That said, from my humble point of view... Premake is very easy to extent, modify and the community is very receptive of changes and merging Pull Request. So looking at the above 4 year old quote, if the person had contributed his extensive modifications, we could have all benefited from it, and maybe Premake would have been even more mature then it is today.

So, the comment is from 4 years ago, in that time a lot has happened. Blizzard Entertainment for example (where I currently work) decided to give it a shot, and we have heavily contributed to many areas of improvement. Other companies have done similar things. Starcraft 2 is a project of over 18000 files distributed over 163 projects (just looking in VS2015 right now). And Premake takes about 34 seconds to generate that solution. That isn't great, and we're certainly very interested in performance improvements, however it's also not bad enough that we are actively looking at that right now. It's one of those things where we're in "some day it would be nice" mode.

Mind you, premake scales almost linearly based on number of files * number of configurations * number of platforms. SC2 has 4 configs, 2 platforms, and about 18000 files. Premake times go down significantly if we only generate the solution for 1 config, for which we added a command line option in our premake scripts.

If we miss a feature, such as LTO, we add it, make a pull request, get it reviewed, and merge it. Now everyone has LTO. Which seems a more responsible approach then complaining about it on Reddit.

Anyway, to respond to the specific issues:

• Secondly, Premake has no way of properly incorporating file-specific build steps.

filter { 'files:specificfile.cpp' }
   buildcommands { "do something here" }

Custom Build Commands

• The generated Makefiles are far from efficient too

There is always room for improvements, suggestions are welcome.

• That's all down to weak choice of O(N) or even O(N2) algorithms in the Lua code.

Specific examples would have been nice, so these could have been fixed. That said, we do have some modifications here at Blizzard to the "getvpath" implementation which gave us a 4x performance improvement in cases where the vpaths API is used extensively.

• if you want to incorporate LTO or PGO into your build, Premake has nothing for you.

These are generally just compiler options, so Premake always has the buildoptions and linkoptions API's to just send raw options to the compiler or linker. But the premake override API, allows you to inject code anywhere and everywhere, and so the ability to completely customize the build output is very powerful, adding LTO if Premake didn't already have it, would have been 5-6 lines of lua.

[starkos]

Thanks for taking the time, Tom. We should save your response for the next time that Reddit post comes up.

Marking as closed.

[tvandijck]

And just to add... I'm not sure what a "complex project" really entails in everyone's opinion. However in my opinion Starcraft 2 as well as Heroes of the Storm are reasonably 'complex', in that they are not pure C/C++ project, they are reasonably 'large', build across multiple platforms, and employ several languages and tools during the build process.

[dumblob]

@tvandijck I've almost lost my breath. Thank you very much for a detailed answer! This answers my question and proves what I believed in already several years ago - Premake ecosystem is a really good choice.

And yes, I consider Starcraft 2 and Heroes of the Storm as definitely complex projects.