Work on dbuild 0.9.0, stage 3/5

[cunei]

This is the second part of the sbt plugin support code. It fixes dependency rewiring, implements the actual building, cleans up old code, and extends further information so that it is applicable to multiple build levels.

[jsuereth]

Ok, so It's WAYYYYY too late to put a stop on the build-level exposed throughout dbuild, but I do think that design is a bit flawed and we should have reworked it. We need to find ways to have these conversations earlier so that the code doesn't exist before the design discussion happens, and we can correct early. Right now, I think we should spend the next few weeks undoing the notion of build levels from showing up in the metadata, and instead find another terminology, or leverage spaces (which fits better the model).

I need to go run some errands, but I'll finish reviewing shortly.

[cunei]

The concept of "levels" is tied to how sbt works. First level=main, second=plugins, third=plugins of plugins, etc. Since there are an indeterminate number of nested builds in sbt, there would have been no point in describing them using labels like "main", "plugins", etc., unless you wanted to artificially limit code that can by its nature be generic and apply to all nested builds within sbt. I debated at the beginning on whether to describe the situation using flat sequence-like metadata to refer to these universes, or using nested records, or again tagging specially the metadata for plugins only. After trying the various alternatives, the simple sequences seemed far, far easier to manage for the end user, so I am fairly sure I want to stick to this choice for now. Notably, each level is bound to exactly one space, which described where (for each level) live the incoming artifacts used during build. Hence, the rematerialization can simply work in the future by keeping one rematerialized repo per space, and pointing the level repo subdirs to the correct ones, simply by changing BuildDirs.localRepos() accordingly.

[jsuereth]

Right, but I'm actually not positive that (a) we need to support more than just a plugins space, (b) the additional multiple levels is worth the complexity burden.

If you think the end-result for users is better, I'll trust to that for now, but it feel fundamentally an issue when multiple things rely on implicit values i.e. data coherence is not high.

Did you run into a project that had more than "build" and "plugin-build" you had to wire?

[jsuereth]

Ok, so generally looks ok to merge to me. Again, regarding architecture/direction we may fundamentally agree, but I still haven't seen the end product. Lots of nice fixes in here for future-ish things, like custom repository lists. Also the fragmenting of build levels is a complicated aspect, but I worry the notion of levels actually hinders our ability to improve/change later and doesn't feel like it's uniform with the rest of dbuild's architecture. i.e. it feels like a unique complication of sbt that has bled throughout the entire architecture. BUT, I still haven't seen the final result, so take that view with a grain of salt.

[cunei]

It depends on the desired complexity of the configuration file. I no longer have all the logs so I don't know how many levels the various projects use, but I'll restart the build and will find out. Code that needs to work for 2 environments can easily work for n, so there is not really an additional complexity. Actually I found that the genericity simplified the code and made it more systematic and robust, in the end. I'm merging now, and will prepare the next chunk shortly.