Closed mabre closed 6 years ago
Yes, I know this error from the parser (frege.compiler.grammar.Frege), where I also have those huge tables.
I've thought about supporting this better by special handling of list literals that have only other literals in them. For example, making them into an array in an extra method. Still, there is a limit even for the size of such an array in Java.
Meanwhile, the easiest thing you can do is to transform
foo = [1,2,3,4,...,123456]
into
import Data.JSON
foo = unJust (parseJSON
"[1,2,3,4,....,123456]"
:: Maybe [Int])
Unfortunately, since we don't have mult-line string literals, the string must be all on one line, or constructed from multiple strings with (++).
In the Frege compiler, I encode the parser tables in strings like so:
> foo = (map ord . unpacked) "\U1234\U4325\U...."
When compiling this code with Frege 3.23.451 which contains some big lists (automatically generated by the lexer generator Alex), I get this error when
javac
is called:This happens because all functions which return a list are translated to class variables which will hit the byte code limit of 65,535 byte. A workaround is giving those functions an unused parameter, such that those functions get translated to java functions.
When compiling the same code with Frege 3.24.100, I run out of heap space (set to 2 GB):
The generated Java file contains 328,532,791 characters. After removing repeated spaces using
sed "s/ +/ /g"
, it has only 2,035,731 characters and I get thecode too large
error again:It would be nice if the compiler could find out if the generated static code is too large and put it into functions.
[btw: The original code only had two functions with two very big lists. When trying to compile that file with 2 GB heap space I get this error after 10 minutes:
But I think preventing these out-of-memory situations in the compiler might be hard/impossible.