FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

edouard-lopez commented 7 years ago

Issue

I'm a gentledev i: [x] fully read the README recently [x] searched for existing issues [x] checked i'm up to date with the latest version of the project

Expected behavior

Completing the task.

Actual behavior

I run the my script on a directory with 41214 SVG files and got the following error

function buildFont(config) {
  return webfont({
    files: config.inputFiles,
    fontName: config.fontName
  })
    .then(content => content)
    .catch(err => {
      console.log(err);
    });
}

Error

node --require babel-core/register ./index.js
already exists                                 

<--- Last few GCs --->                         

  232006 ms: Mark-sweep 1350.0 (1406.9) -> 1349.8 (1406.9) MB, 574.3 / 0.0 ms [allocation failure] [GC in old space requested].
  232583 ms: Mark-sweep 1349.8 (1406.9) -> 1349.8 (1406.9) MB, 576.6 / 0.0 ms [allocation failure] [GC in old space requested].
  233164 ms: Mark-sweep 1349.8 (1406.9) -> 1349.8 (1403.9) MB, 580.3 / 0.0 ms [last resort gc].
  233745 ms: Mark-sweep 1349.8 (1403.9) -> 1349.7 (1403.9) MB, 580.3 / 0.0 ms [last resort gc].

<--- JS stacktrace --->                        

==== JS stack trace =========================================

    2: arguments adaptor frame: 2->1           
Security context: 0x4e54c9cfb39 <JS Object>    
    3: round [/data/projects/hanzi-pinyin-font/node_modules/svg-pathdata/src/SVGPathData.js:12] [pc=0x26e3767b3b09] (this=0x217c839e5e99 <an SVGPathData with map 0x375d0749d7b1>)                                 
    4: arguments adaptor frame: 1->0           
    5: /* anonymous */(aka /* anonymous */) [/data/projects/hanzi-pinyin-font/node_modules/svgicons2svgfont/src/index.js:336] [pc=0x26e376698166]...                                                               

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]                       
 2: 0x109b1dc [node]                           
 3: v8::Utils::ReportApiFailure(char const*, char const*) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [node]
 6: v8::internal::DeoptimizationOutputData::New(v8::internal::Isolate*, int, v8::internal::PretenureFlag) [node] [node]
 7: v8::internal::FullCodeGenerator::PopulateDeoptimizationData(v8::internal::Handle<v8::internal::Code>) [node]
 8: v8::internal::FullCodeGenerator::MakeCode(v8::internal::CompilationInfo*) [node]
 9: v8::internal::Compiler::EnsureDeoptimizationSupport(v8::internal::CompilationInfo*) [node]
10: 0xb538d7 [node]                            
11: 0xb54d4d [node]                            
12: v8::internal::Compiler::CompileOptimized(v8::internal::Handle<v8::internal::JSFunction>, v8::internal::Compiler::ConcurrencyMode) [node]                                                                       
13: v8::internal::Runtime_CompileOptimized_Concurrent(int, v8::internal::Object**, v8::internal::Isolate*) [node]
14: 0x26e374e092a7                             
Aborted                                        
error Command failed with exit code 134.

Steps to reproduce the behavior

Get font generator repo:

git clone git@github.com:parlr/Hanzi-Pinyin-Font.git && cd Hanzi-Pinyin-Font
git check out  9b7bd1c

Get data extractor repo:

git clone git@github.com:parlr/chinese-data.git && cd chinese-data

Extract chinese data:

echo $'glyph\truby' > src/codepoint-ruby.tsv
awk '/U+/ && /kMandarin/{print $1"\t"$3}' src/Unihan_Readings.txt >> src/codepoint-ruby.tsv
./node_modules/.bin/csv2json --tsv ./src/codepoint-ruby.{tsv,json}

Generate font

cp ./src/codepoint-ruby.json ../src/data.json
cd ../  # back to Hanzi-Pinyin-Font
npm start

Debugging informations

node -v result:
```
v6.10.1 
```
npm -v result:
```
3.10.10
```

Question

How can I solve this?

related:

Stackoverflow: Font generation with webfont: FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
itgalaxy/webfont#54

NaridaL commented 7 years ago

Clearly a memory leak of some kind; While the last stack frame is indeed in this project, it's not necessarily the culprit. Have you analyzed the heap? If this isn't a task that needs to be run regularly, upping the memory limit might also be a solution.

edouard-lopez commented 7 years ago

Should not be a common task, but I

upgraded to NPM@4
raise the limit to --max_old_space_size=8192

and it still crashes.

NaridaL commented 7 years ago

Considering the linked issues have all been closed, I'm going to close this too.

edouard-lopez commented 7 years ago

For clarity, I solved my issue with following command (see commit):

node \
    --max_old_space_size=8192 \
    --optimize_for_size \
    --max_executable_size=4096 \
    --stack_size=4096 \
    --require babel-core/register \
./index.js

nfroidure / svg-pathdata