High memory utilization on 1.0.0.0 (d6b1bcb1)

[Lupus]

amc uses gigabytes of memory to compile ~1k sloc codebase. I didn't run into this issue with 0.7. My VM has 4 GB of ram. It does not seem to be connected with specific constructs, just commenting out portions of code make memory usage go down until it finally manages to compile within 4GB that I have.

[Lupus]

To compare, 0.7 uses less that 100MB of ram for the same codebase and runs much faster.

[SquidDev]

I can't think of any obvious changes in the compiler which would cause such a blowup - I will do some profiling later today.

The standard library definitely has some impact on this (assuming you're using it). Using the compiler from 662992a8, and running with time amc +RTS -s -RTS compile load-prelude.ml:

• bbe61c668083400c435d0e1f8d6ea33abc3b3c88: 0.91s, 1,064,830,280 bytes allocated.
• 662992a8afeca24dc3ea4f56cfb24f962bd4102d: 1.43s, 1,611,497,424 bytes allocated.

In both cases, GC amounts to ~40% of runtmie, which is a little worrying. Optimisation is something we really need to work on. That said, the maximum residency is never more than 15MB, so curious that it's not compiling on a 4GB machine.

[SquidDev]

@Lupus Would you be able to run amc compile with the --time flag (e.g. amc compile --time timings your_program.ml) and post the resulting file as a gist? Would be interesting to see what the slow down is. I realise this doesn't cover memory usage, but I suspect the two are related.

[SquidDev]

I fixed a rather embarrassing case of exponential blowup of the type-class trie when combining environments (see b4d4784f21501cefde598befc2e3531e6b26c4c7), which has dramatically improved performance in some cases (40s → 3.7s). It might be worth trying that version and seeing what, if any, improvements it makes. Doesn't really seem to change loading the prelude at all, but you might be lucky :).

[Lupus]

Cool, I'll give it a try when I get back to the project that uses amulet. Thanks!