search

sharedstreets / sharedstreets-js

SharedStreets (Node.js & Javascript)
https://sharedstreets.io
MIT License
78 stars 25 forks source link

JavaScript heap out of memory for large networks #68

Open hbruch opened 4 years ago

hbruch commented 4 years ago

Matching a larger network, I got a heap out of memory exception (see below).

I wonder, if there is a better way than adding a hardcoded --max-old-space-size=xxx to bin/run like that:

#!/usr/bin/env node --max-old-space-size=8192

Exception without added --max-old-space-size:

<--- Last few GCs --->

[56340:0x110000000]    16976 ms: Scavenge 1355.0 (1425.8) -> 1348.9 (1444.3) MB, 4.6 / 0.0 ms  (average mu = 0.177, current mu = 0.054) allocation failure
[56340:0x110000000]    17950 ms: Mark-sweep 1363.8 (1444.3) -> 1350.8 (1436.8) MB, 962.7 / 0.0 ms  (average mu = 0.130, current mu = 0.057) allocation failure scavenge might not succeed
[56340:0x110000000]    17971 ms: Scavenge 1364.5 (1436.8) -> 1352.5 (1439.8) MB, 4.3 / 0.0 ms  (average mu = 0.130, current mu = 0.057) allocation failure

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x38d32c7dbe3d]
    1: StubFrame [pc: 0x38d32c7934b0]
Security context: 0x3259ca21e6e9 <JSObject>
    2: toObject [0x32592a87ed71] [.../.config/yarn/global/node_modules/sharedstreets/node_modules/sharedstreets-pbf/proto/sharedstreets.js:~2595] [pc=0x38d32cbd63ab](this=0x3259033a7c01 <JSFunction SharedStreet.

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
lmz commented 2 years ago

I've had luck setting this through the environment variable, NODE_OPTIONS

e.g.

export NODE_OPTIONS=--max_old_space_size=8192