search

MLton / mlton

The MLton repository
http://mlton.org
Other
962 stars 127 forks source link

mlton compiler writing the resulting binary executable very slow (disk mounted over fast by high-latency connection) #556

Closed jtrint closed 7 months ago

jtrint commented 7 months ago

I use mlton with my working directory mounted over a fast but relatively high-latency connection (sshfs). Compiling the project I now work with takes 30 seconds CPU time and 3 minutes real time, apparently because of mlton writing the resulting 10 MB executable to disk in small pieces. I can eliminate the problem by writing the executable to my local /tmp and then copying the result, but this is not the nicest solution. I have not had any similar issues with other software over the same sshfs connection.

MatthewFluet commented 7 months ago

MLton simply invokes the C compiler to link the object files and libraries to produce the executable. You can compile with -keep g -keep o -verbose 2 to see the exact C commands that are used to compile the C and assembly files to object files and to link the object files; that will also show the time spent in the "Compile and Assemble" and the "Link" phases.

MatthewFluet commented 7 months ago

@jtrint Any additional information about the slowdown? As mentioned, MLton just dispatches to the system C compiler to perform the compilation and link steps.

jtrint commented 7 months ago

I checked with the options you suggested.

The very slow phase is after "Link starting". I have half a dozen libraries and archives (compiled C that I access through the FFI, including a BDD library and two SAT solvers). During the slow phase (which lasts a couple minutes), ssh and sshfs both take about 5 per cent CPU, and "ld" once in a while pops up in top with a some percent points of CPU, but otherwise is just waiting for something (= writing/reading through sshfs).

Surprising thing is that another, far bigger SML program, with just a couple of small archives/libraries, goes through the same linking phase almost instantaneously.

This is then not really a MLton issue, and might never show up for a vast majority of MLton users, so I am happy with this as is.

MatthewFluet commented 7 months ago

Thanks for the update; since this doesn't seem to be a MLton-specific problem, I'll close the issue.