pnkfelix / larceny-test

test import of trac db
Other
2 stars 0 forks source link

Speed up the Larceny build process #472

Open larceny-trac-import opened 11 years ago

larceny-trac-import commented 11 years ago

_Reported by: pnkfelix on Fri Sep 14 12:24:08 2007 _ (related tickets: #122 #415 #65 #84 #71)

Some discussion on the [http://scheme-punks.cyber-rush.org/wiki/ ERR5RS] wiki indicated that Larceny builds can take an absurdly long time compared to other Scheme systems.

So Felix is opening this ticket as a point to collect:

The [http://larceny.ccs.neu.edu/dart Dart] autobuild has timing information each night for how long each of the builds took.

Here's the email (after slight edits) I sent to Will on the matter on September 12th, 2007:

On the Twobit pass times: I gathered numbers on SunOS as well as Linux.

As you know, assembly on SPARC takes little time compared to overall compilation time. On SPARC, 9.1% of compilation time is spent in the assembler.

According to the data I gathered, assembly on IA32 takes a significant amount of time, but it is not fair to claim that it takes 75% of compilation time. On IA32, 28.9% of compilation time is spent in the assembler. 21.8% of compilation time is spent within the Sassy assembler itself; the remaining time can be charged to code I wrote to interface with Sassy.

There is a difference between the SunOS and Linux beyond what assembler they use in their compiler, and this probably accounts for the difference in the nightly builds that you used to derive the 75% figure. Each system needs to compile the corresponding files for each of their assemblers. Maybe that was obvious to you, but it wasn't a cost that I was thinking of when we spoke during our meeting.

The reason that this is relevant is that it takes a noticeably long time for us to compile some of the files for Sassy. ([source:trunk/larceny_src/src/Lib/Sassy/opcodes.scm opcodes.scm] is particularly painful to load). Plus we end up effectively compiling several of these files twice: an in-heap compile when we initially load the files into Larceny, and a second compile to a fasl file when we do the (build-larceny-files) step. So we end up getting charged twice for the time spent compiling any particular file in Sassy.

        Elapsed Time in ms
            IA32        SPARC
-----------------------------------------------------
COMPILE:        211,678         184,772
    PASS1:       89,016 (30%)   110,128 (54%)
    PASS2:        1,909 ( 1%)     3,419 ( 2%)
    PASS3:      112,483 (38%)    58,589 (29%)
    PASS4:        4,802 ( 2%)     6,917 ( 3%)
ASSEMBLE:        85,885 (29%)    18,650 ( 9%)

OVERALL:        297,563     203,422

Here is similar data, but broken down according to whether the time is being spent:

                         IA32         SPARC
LOAD-TWOBIT
 * PASS1               41,832       108,975           
 * PASS2                  907         1,318
 * PASS3               78,312        19,321
 * PASS4                2,414         3,292
 * ASSEMBLE            45,021         4,365
BUILD-HEAP
 * PASS1               47,328        61,905
 * PASS2                  964         2,048
 * PASS3               20,454        36,263
 * PASS4                1,969         3,719
 * ASSEMBLE            28,681        13,961
BUILD-FASL
 * PASS1               41,733        49,354
 * PASS2                1,020         1,483
 * PASS3               85,756        22,823
 * PASS4                2,767         3,421
 * ASSEMBLE            56,262         4,958
larceny-trac-import commented 11 years ago

Author: will System builds should be a little faster in v0.963 because Twobit pass3 now limits the number of available expressions.

For example, [source:trunk/larceny_src/src/Lib/Sassy/opcodes.scm opcodes.scm] now compiles in less than 20 seconds on Will's MacBook Pro, spending about 8 seconds in pass3 and 8 seconds in assembly.

Most of the pass1 problem had been addressed in v0.961 or earlier, although quasiquote is still slow.

In v0.963, the main problem appears to be the assemblers, and they are okay in R5RS mode unless the procedure they are compiling is very large. See WhyTwobitIsSlow.