Feature Request: build static libraries concurrently

[MrcSnm]

System information

• dub version: 1.34.0
• OS Platform and distribution: All
• compiler version 2.105.0

currently, dub builds are synchronous and this can cause a lot of times lost since it is not really required.

I have been investigating this idea myself and it is a matter of making the build system less stateful, not using recursion on buildTarget and avoid using delegates.

This work is possible but also depends on a good strategy of printing #2701

The static libraries doesn't really depends on each other since the linking prices only happens at the executable or shared library.

This could also decrease the time for dub process up to date dependencies so the project as a whole would benefit from it, resulting in possibly better compilation speed than plain DMD.

The reference file is dub/generators/build.d At the method generateTargets

1. Remove recursion code and use stacks instead, it will be a huge gain since it won't do recursion using a function defined inside a method which also is a delegate (I don't really know how could I profile that, but I'm sure this must be bad).
2. Identify leaf dependencies for build them concurrently
3. Use a taskpool(?) for spawning threads and wait for them before returning

This process should at least make compiling way faster since I don't get much CPU usage when I'm using dub, so it could make rebuilds a lot faster (and hopefully make my problem go away)

[rikkimax]

This shouldn't need to be limited to static libraries.

They may lack the link step, but they still have the pre and post build commands, which could complicate matters.

[MrcSnm]

@Geod24 I think this feature is very important and it is conceptually very simple. Since you've been working with dub, would you take a look on it? If you need any implementation help I could give a hand since this could be the best change in dub in at least 3 years :)

[Geod24]

@MrcSnm : I would be happy to have parallel builds in Dub, however I don't think I can get to it in the near future.

My work so far has focused on the package manager side of Dub, rarely on the build part. The build part needs to be re-thought and re-architectured IMO, as it doesn't make sense to build D like one builds C/C++. Having per-package build has been heavily discussed over the year, and having a more frontend-driven process as well (instead of assuming everything under source/ goes).

[MrcSnm]

@Geod24 I understand the implications on why it is questionable having a per package build, but I have saw some advantages on it when we have some packages with heavy meta-programming steps or when they import a module hard to parse(which takes much time). See core.sys.windows.uuid. This single module takes 400ms on any compiler which is just a lot of time.

So, I have a question on how hard would be to separate dub in build vs package manager. There are plenty of people which likes the package manager part of dub but hates it as a build system. I personally like its simplicity but I think it could be a lot less complex.

I would like to work on that but unfortunately I have plenty of work to do still in my projects