Lake generates several intermediate files and leaves them lying around

My lakefile is:

c.shared{'lualdap', src='src/lualdap.c', needs='lua', odir=true}

After running Lake, I have following files lying around:

lakefile.spec
lualdap.d
lualdap.o
_t_.spec

When I specify odir=true, the situation is a little bit better, as .d and .o are now in the release/ subdirectory.

Several remarks on this:

t.spec seems to duplicate the contents of lakefile.spec
Can these *.spec files go into some subdirectory?
Would be nice if Lake would support out-of-source builds, where I call it from some directory X, pointing it to the lakefile and sourcecode in directory Y. There is already a -f parameter, but the paths are still treated as relative to the working directory, instead of relative to the lakefile.
Can odir specify a directory, or just be true/false? That is not clear to me from the docs:

odir -- output directory; if true then use 'debug' or 'release'; prepends PREFIX

I figured out that PREFIX is a global variable, but then its description confused me:

PREFIX - (empty string. e.g. PREFIX=arm-linux makes CC become arm-linux-gcc etc)

That is what other buildsystems refer to as TARGET (or CHOST/CTARGET in case of autotools)...

Can odir be the default? Or a behaviour similar to odir, where by default the whole build process takes place in a separate dir? A good default for its name might be "build".

I agree that Lake currently leaves too much mess around. The default clean target goes for the files associated with each file target, which isn't satisfactory. First, there are other files (especially the junk left by MSVC) - I do try to eliminate the .d files associated with .o files. Second, it's all or nothing - maybe 'clean' should leave the ultimate outputs alone (be they libraries or executables) and just clean out the intermediate files. Then we have another word that means 'clean everything', maybe 'really_clean' ?

There is both _t.spec and lakefile.spec because Lake is trying to determine if a build changed - i.e. force a rebuild if a build parameter changes. Not entirely sure whether it's still working ;) In any case, maybe these guys should go into the build directory.

Making odir default to true seems like a good default. The default names are 'debug' or 'release' depending on whether called with -g or DEBUG=1. If I run Lake with lake PREFIX='arm-linux' then the default becomes arm-linux-release etc to allow multi-platform builds. You can indeed set odir to an actual directory, and that is used as is, with no prefix. In all cases Lake will create the directory if it doesn't already exist.

Yes, out-of-directory builds are a good thing, and something I will now work on. Maybe a distinction between odir (alias output_directory) for intermediate files and bdir (alias bin_directory) would be useful ?

Naming things is hard, as always ....

autotools has clean and distclean for this purpose. The former shall clean all generated files, the latter return the source directory to the state after unpacking the tarball (normally only the build configuration). CMake goes a different route and provides only clean, but tells you to use a build directory. Now what would be really great would probably be to encourage the user to use a build directory (by making that the default behaviour) and in addition stuffing the files used only by Lake (I understand these are the *.spec files) into a subdirectory of the builddir (even if =srcdir), conveniently named "LakeBuild" or similar.

Maybe adding an alias to odir, named builddir, would be nice for self-documentation purposes.

What would bindir contain? Only the primary targets of the Lakefile? How do you determine which those are, i.e. what the user actually wants to work with, especially when it comes to custom rules/targets? Maybe it would instead be better to provide install support, and make it easy for the user to install the files he needs into a bindir of his own?

The question about primary targets is important because that's exactly the limitation of auto-generated clean targets. Naively I would regard the targets created by program/shared/etc as primary, but in general it's unwise to try to be telepathic - it irritates users. Currently, at least the model is straightforward - everything goes to odir (aka builddir); if you want your libs/exes to go somewhere else then it should be explicit (Lake can ensure that the directory does exist). So is the suggestion that by default program/shared/etc put their targets in a directory other than odir, if the names do not have a path part? Certainly (if I pretend to be a naive user) I would be happy with all the .o files going into release by default, but I would expect the lib/exe to end up in the current directory.

(That rule would mess with having parallel debug/release/platform builds by default)

More elaborate projects tend to put the result in subdirectories like libs, bin and so on.

Install support - I did think of that, it's a can of worms. Unix-style operating systems do have conventional places for libs and exes, but Windows is a mess and OS X is ... different.

Lua install support is easier (since Lake knows about the local Lua installation) but held off because something like LuaRocks does that well and manages the dependencies. I actually wrote a custom Lake build method for LuaRocks but decided against including it in the Lake rockspec; perhaps its time has come.

Re: Clean

I also think that magically figuring out what the user wants is a bad idea.

Hence my suggestion to use odir (with alias builddir), where everything that is being build goes into. That includes binaries and libraries as well as intermediate files like *.o. I would not try to figure out the primary targets and put them into a different dir, since the user might define own rules and targets where this automagic still has to work, or he might want to use the static library we built as a dependency of his program in another project, etc.

The files used by Lake internally (e.g. _t_.spec and lakefile.spec), should go into a subdirectory of the builddir, possibly named LakeDir or similar.

The suggestion is further to not try to do magic with the clean target (like inventing a distclean or really_clean target), but to push the user to use builddir by setting a default builddir="build" (instead of builddir=".", i.e. builddir=srcdir, used by other buildtools like CMake or autotools). You can then advise the user to run rm -fr ${builddir} to get rid of the unwanted stuff. Further such a dir can easily be excluded in a .gitignore file or similar.

Re: Install

The projects which want to use libs and bin, could choose on their own which target shall go into which directory, if Lake would support installation.

They could add lake.install(my_library), and get exactly what they need, without any unexpected automagic. If they set lake.cfg.libdir="$(srcdir)/bin", their binaries could still be in the bin subfolder, without Lake needing to read their mind.

On Linux you can specify useful defaults for lake.cfg.libdir and so on (e.g. "$(lake.cfg.prefix)/lib"). On Windows you could just set the default to $(srcdir)/LakeInstall for all types, since the user will probably need to manually copy it out of the destination dir anyway. I don't know about MacOSX.

I just had a quick look at LuaRocks, and it seems to be a simple buildsystem on its own, i.e. it does the same job as Lake, just for very simple cases. If LuaRocks can be easily extended to support calls to Lake, doing that might be a good idea.

In general ...

... I think it is good to have a look at autotools, as it does many things right (just with lots of legacy luggage and difficult scripting/extensibility). Take for example cross-compilation, which no other buildtool (to my knowledge) supports as nicely as autotools. Or take optional dependencies, configuration flags, destination directories, or even checks for libraries and such things. Combine it with pkg-config (and possibly LuaRocks or similar) and you get something quite flexible and still straightforward.

Lake no longer uses *.spec files, since they really were not much use at all - they were intended to solve the "files unchanged but different flags" problem but never really worked. So there's less junk now ;)

As for LuaRocks, it's entirely possible to use it as a backend. I'll update the old lake on the LuaRocks repo and then people can write rockspecs which have Lake as a dependency.

Making odir a default thing is attractive, I must say. I've shied away from built-in install support because even on Posix it's a bit of a mess and Windows is a totally different beast.

stevedonovan / Lake