JavaScript heap out of memory when generating imports for @arcgis/core/views/MapView.js

[zachsa]

This import:

import MapView from '@arcgis/core/views/MapView.js'

Results in an out-of-memory JS error:

<--- Last few GCs --->

[22772:0x69d4740]    69519 ms: Scavenge 2014.6 (2075.0) -> 2011.9 (2076.5) MB, 16.0 / 0.0 ms  (average mu = 0.620, current mu = 0.344) allocation failure
[22772:0x69d4740]    69554 ms: Scavenge 2016.1 (2076.5) -> 2013.5 (2079.5) MB, 21.0 / 0.0 ms  (average mu = 0.620, current mu = 0.344) allocation failure
[22772:0x69d4740]    70437 ms: Scavenge 2019.4 (2079.5) -> 2016.1 (2096.7) MB, 866.6 / 0.0 ms  (average mu = 0.620, current mu = 0.344) allocation failure

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb09980 node::Abort() [node]
 2: 0xa1c235 node::FatalError(char const*, char const*) [node]
 3: 0xcf784e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xcf7bc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xeaf465  [node]
 6: 0xebf12d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 7: 0xec1e2e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 8: 0xe8336a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
 9: 0x11fc0b6 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
10: 0x15f0b19  [node]

Running the generator with --max-old-space-size=4096 also crashes - it just takes longer! This does work on the generator site, but it freezes the browser for a while: https://generator.jspm.io/#U2VhYGBkDM0rySzJSU1hcEgsSk7PLNZPzi9KdTDRMzLRM9DNS60o0TMyMDIyMDU01i/LTC0v1vdNLAgDMvSyigEB0qFYQQA

[guybedford]

Did you try increasing the heap size? This is possible in Node.js via the --max-old-space-size option.

[zachsa]

yes - to 4096, doesn't work. I've tried increasing it again, and it's still running

[zachsa]

The library is very large - 43MB or so. Here is the package json in case it helps:

{
  "name": "@arcgis/core",
  "version": "4.24.7",
  "homepage": "https://js.arcgis.com",
  "description": "ArcGIS API for JavaScript: A complete 2D and 3D mapping and data visualization API",
  "keywords": [
    "ArcGIS",
    "javascript",
    "map",
    "3D",
    "2D",
    "visualization",
    "analytics",
    "spatial",
    "data-driven",
    "gis",
    "Esri"
  ],
  "license": "SEE LICENSE IN copyright.txt",
  "browser": {
    "buffer": false,
    "crypto": false,
    "fs": false,
    "path": false,
    "process": false
  },
  "dependencies": {
    "@esri/arcgis-html-sanitizer": "~2.10.0",
    "@esri/calcite-colors": "~6.0.1",
    "@esri/calcite-components": "1.0.0-beta.82",
    "@popperjs/core": "~2.11.5",
    "focus-trap": "~6.9.4",
    "luxon": "~2.4.0",
    "sortablejs": "~1.15.0"
  },
  "type": "module"
}

[zachsa]

I'm trying to figure out what can cause this - specifically whether it's a user error, an esri error, or a generator error. Do you have any guidance @guybedford ?

[guybedford]

I think this is actually just a kind of performance cliff issue, and makes a great case for working through some performance optimization. I'm looking into it.

[guybedford]

I was able to fix this in https://github.com/jspm/generator/pull/151, turns out there was even a perf comment on the right line.

[zachsa]

I just tested via cloning the repo, and I get the same error:

<--- Last few GCs --->

[4627:0x68cd740]    60599 ms: Scavenge 2020.2 (2080.2) -> 2016.7 (2081.2) MB, 12.4 / 0.0 ms  (average mu = 0.690, current mu = 0.316) allocation failure 
[4627:0x68cd740]    60619 ms: Scavenge 2021.6 (2081.2) -> 2019.4 (2084.5) MB, 12.5 / 0.0 ms  (average mu = 0.690, current mu = 0.316) allocation failure 
[4627:0x68cd740]    61172 ms: Scavenge 2024.4 (2084.5) -> 2021.4 (2102.5) MB, 548.8 / 0.0 ms  (average mu = 0.690, current mu = 0.316) allocation failure 

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb09980 node::Abort() [node]
 2: 0xa1c235 node::FatalError(char const*, char const*) [node]
 3: 0xcf784e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xcf7bc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xeaf465  [node]
 6: 0xebf12d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 7: 0xec1e2e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 8: 0xe8336a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
 9: 0x11fc0b6 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
10: 0x15f0b19  [node]

[guybedford]

While you are waiting for the release, one approach you could use is https://www.npmjs.com/package/patch-package in the mean time.

[zachsa]

Thanks - it does actually work. that was a testing error. I'm going to copy the dist into my project until the release