Open maelvls opened 7 years ago
I just tested on a linux box with 64GB of RAM and ghc 8.0.2: at the end of the build, koka is at 20GB of resident memory (whatever this means). Also, 1TB of virtual memory is asked:
Nevertheless, the 20GB memory footprint is propably due to a memory leak somewhere (ghc? koka?)
Update: the 1TB of virtual memory seems normal on 64-bits arch according to the official 8.0.1 notes.
Update2: this problem also seems to exist on other projects like ghc-mod
(see issue here) and seems to be related to ghc >= 8. This bug might not exist on ghc 7.
Update3: I just built koka-0.6.0-dev
using ghc 7.10.3. But same problem: when compiling madoko, koka-0.6.0-dev
takes a huge amount of memory. @daanx, what version of ghc are you building koka-0.6 on? Maybe I am doing it all wrong?
Hi,
I did some digging using GHC 7.10.2. It seems that some substitute
function has an issue and never de-allocates for some reason. I think it relates to the substitute
in koka-0.6/src/Type/TypeVar.hs
.
Here are my steps to produce this graph (I followed a guide):
First, compile koka using the profile mode:
cd koka-0.6 git checkout v0.6.0-dev jake variant=profile
Then, compile madoko
using the koka compiler we just compiled:
cd madoko ../koka-0.6/out/profile/koka-0.6.0-dev -isrc -i../koka-0.6/lib -c -olib --outname=madoko -v main +RTS -hc
You can ctrl+c when you feel it's enough; then
hp2ps -c koka-0.6.0-dev.hp
which will create a postscript fill koka-0.6.0-dev.ps
. To convert it to png:
convert -density 300 -rotate -90 koka-0.6.0-dev.ps koka-0.6.0-dev.png
Here is koka-0.6.0-dev.hp.zip
Reproduced with ghc 7.10.3 too. My version sees the substitute/|->/subCom...
hill divided into many colors. The -hr
graph gives a bunch of substitute
. The -hm
graph unsurprisingly shows all TypeVars. The -hd
graph blames a certain closure Data.Map.Base.sat_sjGv
, but that is meaningful only for my current distribution of ghc; there's some bytes in the trace info (DWARF?) that says ->>Map^@^@<map>
after it. -hy
shows a huge *
in the postscript graph.
Which "subCom..." is it though? subCommon? subCompose?
Hi all, apologies for the late reaction :-) As a quick fix, you can restart "jake" a few times and each time it should make more progress.
One problem that causes this is that Madoko currently relies on a rather old version of the Koka compiler (0.6) that was before Koka started to have algebraic effects. I hope to upgrade Madoko to a newer version soon as it has been lagging for too long.
Secondly, there is indeed some kind of space leak inside the compiler but I have as yet not been able to fix it yet -- but the space profile up there gives a big hint! I guess the problem is in Type/TypeVar/subCompose
-- perhaps a few seq
calls will alleviate some of these problems?
For what's it worth, I saw this on macOS, too. Memory consumption went over 18.3 GB.
Hi,
While trying to hack the excellent madoko processor (see #6), I noticed that the memory needed for running
koka-0.6.0-dev
is huge (using ghc 8.0.2). I only have 4GB of RAM here, and koka wants over 3.8GB during the build:If I let the process running, the whole system freezes. My workaround is simply to launch
koka
, kill it as soon as it reaches ~2GB and the relaunch it and so on.Do you have any idea what could be causing that?
Thanks!