CPU/Ram usage and compilation times

[Dav1dde]

Compiling this piece of code:

import ddl;

import std.string : toStringz;
import deimos.awesomium.awesomium; 

void main() {
     auto awe = loadLibrary!(deimos.awesomium.awesomium)("libawesomium-1.6.5", false);

     assert(awe.awe_initialize_default == null);

     awe.loadFunction!("awe_initialize_default")();
 }

Takes several minutes to compile (dmd 2.061), uses up one core completly and also uses more than 2GB ram.

awesomium deimos bindings.

[jkm]

How long does running $ make config=tests take?

Unfortunately compile time features can take a long time with dmd. I don't know what I can do within ddl to mitigate the problem. Profiling it is kind of difficult. Would it help if it was possible to store the generated code?

[Dav1dde]

I can't use premake4 for the makefile and I can't get the linked version to work. But I don't think that it's my computer which takes that long.

[jkm]

Would it help if it was possible to store the generated code? Then you may only need to compile once?

[jkm]

The performance is here the same.

[Dav1dde]

Mh, storing the compiled code, this could work, and instead of recompiling it, using import(path)?

[jkm]

No. I have to do something to support it I think. Just need to know whether it would help in your case.

[Dav1dde]

If it makes things faster and using less ram, it will definitly help. But I am at the moment in no real need to have ddl working, it woule be a great feauture though (using a combination of derelict, deimos and linking with dynamic libs atm.)

[jkm]

I think it won't. You would have to it once if the code does not change. I can look at the code to find places that make dmd's life hard. But this is very difficult as I have no idea where dmd spends its resources.

[Dav1dde]

The problem is, with this compile times ddl is basically dead for me. Doing it once isn't a problem, since I can pack the generated code with my application.

[jkm]

I think I have found the problem. I'm looking for a fix right now.

[jkm]

I managed to reduce the resource consumption on my machine. Can you comment out the static assert in loadFunction and unloadFunction? Since these result in quadratic compile time in the number of extern(C) functions. I used dmd 2.060.

[Dav1dde]

This makes sense, staticIndexOf finds the index in a recursive way so if there is a long list of arguments, the memory consumption can grow extremly. I tried it and it's still slow, uses one core completly up and takes up to 25% memory (2GB)

[jkm]

So basically no change. Sadly I do not think that I am doing something horribly inefficient. Further the behavior that dmd uses too much memory and time for compile time features is also known for other projects (see e.g. https://github.com/PhilippeSigaud/Pegged/issues/98). Currently I have no idea how to attack this. Sorry.

[Dav1dde]

Only solution is probably waiting... Seems like I have to do it manually. Thanks for your help!