marionebl / svg-term-cli

Share terminal sessions via SVG and CSS
MIT License
3.48k stars 116 forks source link

Fatal error: ineffective mark-compacts near heap limit allocation failed - Javascript heap out of memory #62

Closed Kabouik closed 4 years ago

Kabouik commented 4 years ago

I am trying to convert a rather long (~80 s) cast to svg, but I am hitting a memory issue:

mathieu@xiaomimi-solus ~ $ npx svg-term-cli --cast=336443 --out asciicast-nnn.svg --window
npx: installed 201 in 9.255s

<--- Last few GCs --->

[548208:0x396a570]    92078 ms: Scavenge 1786.8 (2059.9) -> 1775.1 (2062.1) MB, 16.7 / 0.0 ms  (average mu = 0.268, current mu = 0.222) allocation failure 
[548208:0x396a570]    92191 ms: Scavenge 1792.9 (2062.1) -> 1781.2 (2064.4) MB, 18.1 / 0.0 ms  (average mu = 0.268, current mu = 0.222) allocation failure 
[548208:0x396a570]    92296 ms: Scavenge 1799.4 (2064.4) -> 1787.5 (2066.6) MB, 18.4 / 0.0 ms  (average mu = 0.268, current mu = 0.222) allocation failure 

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x1296b99]
Security context: 0x25bdbbac08a1 <JSObject>
    1: p [0x2485eba27579] [/home/mathieu/.npm/_npx/548208/lib/node_modules/svg-term-cli/node_modules/load-asciicast/index.js:~2044] [pc=0x23eff06629e](this=0x3d169de03061 <JSGlobal Object>,0x3f7693688399 <JSObject>)
    2: /* anonymous */ [0x1c49ead282d1] [/home/mathieu/.npm/_npx/548208/lib/node_modules/svg-term-cli/node_modules/load-asciicast/index.js:~1204] [pc...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Writing Node.js report to file: report.20200604.131906.548208.0.001.json
Node.js report completed
 1: 0x91b658 node::Abort() [svg-term]
 2: 0x91caa1 node::OnFatalError(char const*, char const*) [svg-term]
 3: 0xa93a32 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [svg-term]
 4: 0xa93cee v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [svg-term]
 5: 0xc20e35  [svg-term]
 6: 0xc33dac v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [svg-term]
 7: 0xc34aab v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [svg-term]
 8: 0xc36f3c v8::internal::Heap::AllocateRawWithLightRetry(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [svg-term]
 9: 0xc37261 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [svg-term]
10: 0xbfc9e9 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType) [svg-term]
11: 0xf33f42 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [svg-term]
12: 0x1296b99  [svg-term]
Aborted

Is the conversion for that kind of long cast totally unrealistic? Would the end file be huge if there are not too many colours in the original cast? What I would like to do is embed the animation in a README page for demo, which is not doable with the base asciinema cast format.

Kabouik commented 4 years ago

Turns out I get the same user when using --to 2000 to convert only the first two seconds of the cast, so maybe it's not related to its overall duration?

Kabouik commented 4 years ago
cat report.20200604.141534.568642.0.001.json 

{
  "header": {
    "reportVersion": 1,
    "event": "Allocation failed - JavaScript heap out of memory",
    "trigger": "FatalError",
    "filename": "report.20200604.141534.568642.0.001.json",
    "dumpEventTime": "2020-06-04T14:15:34Z",
    "dumpEventTimeStamp": "1591272934288",
    "processId": 568642,
    "cwd": "/home/mathieu/.npm-global/bin",
    "commandLine": [
      "node",
      "./svg-term",
      "--cast",
      "336443",
      "--out",
      "asciinema-nnn.svg",
      "--window"
    ],
    "nodejsVersion": "v12.15.0",
    "glibcVersionRuntime": "2.31",
    "glibcVersionCompiler": "2.29",
    "wordSize": 64,
    "arch": "x64",
    "platform": "linux",
    "componentVersions": {
      "node": "12.15.0",
      "v8": "7.7.299.13-node.16",
      "uv": "1.33.1",
      "zlib": "1.2.11",
      "brotli": "1.0.7",
      "ares": "1.15.0",
      "modules": "72",
      "nghttp2": "1.39.2",
      "napi": "5",
      "llhttp": "2.0.4",
      "http_parser": "2.9.3",
      "openssl": "1.1.1d",
      "cldr": "36.0",
      "icu": "65.1",
      "tz": "2019c",
      "unicode": "12.1"
    },
    "release": {
      "name": "node",
      "lts": "Erbium",
      "headersUrl": "https://nodejs.org/download/release/v12.15.0/node-v12.15.0-headers.tar.gz",
      "sourceUrl": "https://nodejs.org/download/release/v12.15.0/node-v12.15.0.tar.gz"
    },
    "osName": "Linux",
    "osRelease": "4.9.215-155.lts",
    "osVersion": "#1 SMP PREEMPT Wed Mar 4 03:12:02 UTC 2020",
    "osMachine": "x86_64",
    "cpus": [
      {
        "model": "Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz",
        "speed": 2827,
        "user": 212084200,
        "nice": 7000,
        "sys": 53955100,
        "idle": 633288700,
        "irq": 0
      },
      {
        "model": "Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz",
        "speed": 2744,
        "user": 214851000,
        "nice": 6600,
        "sys": 47086000,
        "idle": 37941200,
        "irq": 0
      },
      {
        "model": "Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz",
        "speed": 2762,
        "user": 197990300,
        "nice": 9900,
        "sys": 41912200,
        "idle": 39383200,
        "irq": 0
      },
      {
        "model": "Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz",
        "speed": 2817,
        "user": 182066000,
        "nice": 6800,
        "sys": 36338200,
        "idle": 41540800,
        "irq": 0
      }
    ],
    "networkInterfaces": [
      {
        "name": "lo",
        "internal": true,
        "mac": "00:00:00:00:00:00",
        "address": "127.0.0.1",
        "netmask": "255.0.0.0",
        "family": "IPv4"
      },
      {
        "name": "wlp2s0",
        "internal": false,
        "mac": "90:61:ae:f1:f7:52",
        "address": "192.168.1.12",
        "netmask": "255.255.255.0",
        "family": "IPv4"
      },
      {
        "name": "lo",
        "internal": true,
        "mac": "00:00:00:00:00:00",
        "address": "::1",
        "netmask": "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff",
        "family": "IPv6",
        "scopeid": 0
      },
      {
        "name": "wlp2s0",
        "internal": false,
        "mac": "90:61:ae:f1:f7:52",
        "address": "2a01:cb00:8023:100:21e2:f0aa:c02c:128d",
        "netmask": "ffff:ffff:ffff:ffff::",
        "family": "IPv6",
        "scopeid": 0
      },
      {
        "name": "wlp2s0",
        "internal": false,
        "mac": "90:61:ae:f1:f7:52",
        "address": "fe80::e92e:fa4:3335:f0c",
        "netmask": "ffff:ffff:ffff:ffff::",
        "family": "IPv6",
        "scopeid": 2
      }
    ],
    "host": "xiaomimi-solus"
  },
  "javascriptStack": {
    "message": "No stack.",
    "stack": [
      "Unavailable."
    ]
  },
  "nativeStack": [
    {
      "pc": "0x0000000000a58397",
      "symbol": " [svg-term]"
    },
    {
      "pc": "0x0000000000a59c1f",
      "symbol": "report::TriggerNodeReport(v8::Isolate*, node::Environment*, char const*, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, v8::Local<v8::String>) [svg-term]"
    },
    {
      "pc": "0x000000000091caea",
      "symbol": "node::OnFatalError(char const*, char const*) [svg-term]"
    },
    {
      "pc": "0x0000000000a93a32",
      "symbol": "v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [svg-term]"
    },
    {
      "pc": "0x0000000000a93cee",
      "symbol": "v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [svg-term]"
    },
    {
      "pc": "0x0000000000c20e35",
      "symbol": " [svg-term]"
    },
    {
      "pc": "0x0000000000c33dac",
      "symbol": "v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [svg-term]"
    },
    {
      "pc": "0x0000000000c34aab",
      "symbol": "v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [svg-term]"
    },
    {
      "pc": "0x0000000000c36f3c",
      "symbol": "v8::internal::Heap::AllocateRawWithLightRetry(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [svg-term]"
    },
    {
      "pc": "0x0000000000c37261",
      "symbol": "v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [svg-term]"
    },
    {
      "pc": "0x0000000000bfc9e9",
      "symbol": "v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType) [svg-term]"
    },
    {
      "pc": "0x0000000000f33f42",
      "symbol": "v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [svg-term]"
    },
    {
      "pc": "0x0000000001296b99",
      "symbol": " [svg-term]"
    }
  ],
  "javascriptHeap": {
    "totalMemory": 2162831360,
    "totalCommittedMemory": 2159441368,
    "usedMemory": 1861573112,
    "availableMemory": 92487600,
    "memoryLimit": 2197815296,
    "heapSpaces": {
      "read_only_space": {
        "memorySize": 262144,
        "committedMemory": 32568,
        "capacity": 261872,
        "used": 32296,
        "available": 229576
      },
      "new_space": {
        "memorySize": 33554432,
        "committedMemory": 31196856,
        "capacity": 16759808,
        "used": 2323216,
        "available": 14436592
      },
      "old_space": {
        "memorySize": 2120331264,
        "committedMemory": 2119606952,
        "capacity": 1877074216,
        "used": 1850996528,
        "available": 26077688
      },
      "code_space": {
        "memorySize": 950272,
        "committedMemory": 871936,
        "capacity": 772224,
        "used": 772224,
        "available": 0
      },
      "map_space": {
        "memorySize": 1052672,
        "committedMemory": 1052480,
        "capacity": 832880,
        "used": 832880,
        "available": 0
      },
      "large_object_space": {
        "memorySize": 6631424,
        "committedMemory": 6631424,
        "capacity": 6612416,
        "used": 6612416,
        "available": 0
      },
      "code_large_object_space": {
        "memorySize": 49152,
        "committedMemory": 49152,
        "capacity": 3552,
        "used": 3552,
        "available": 0
      },
      "new_large_object_space": {
        "memorySize": 0,
        "committedMemory": 0,
        "capacity": 16759808,
        "used": 0,
        "available": 16759808
      }
    }
  },
  "resourceUsage": {
    "userCpuSeconds": 150.44,
    "kernelCpuSeconds": 2.79246,
    "cpuConsumptionPercent": 168.387,
    "maxRss": 2263900160,
    "pageFaults": {
      "IORequired": 0,
      "IONotRequired": 915606
    },
    "fsActivity": {
      "reads": 0,
      "writes": 0
    }
  },
  "uvthreadResourceUsage": {
    "userCpuSeconds": 81.3339,
    "kernelCpuSeconds": 1.33621,
    "cpuConsumptionPercent": 90.8463,
    "fsActivity": {
      "reads": 0,
      "writes": 0
    }
  },
  "libuv": [
  ],
  "environmentVariables": {
    "SHELL": "/bin/bash",
    "COLORTERM": "truecolor",
    "XDG_CONFIG_DIRS": "/usr/share/xdg:/etc/xdg:/usr/share",
    "__GL_MaxFramesAllowed": "1",
    "GNOME_KEYRING_CONTROL": "/home/mathieu/.cache/keyring-B8LIL0",
    "HISTSIZE": "1000",
    "I3SOCK": "/run/user/1000/i3/ipc-socket.1478",
    "SSH_AUTH_SOCK": "/tmp/ssh-t5nXJL4KcMSG/agent.1478",
    "NNN_FIFO": "/tmp/nnn.fifo",
    "DESKTOP_SESSION": "i3",
    "SSH_AGENT_PID": "1500",
    "EDITOR": "micro",
    "GTK_MODULES": "gail:atk-bridge",
    "XDG_SEAT": "seat0",
    "PWD": "/home/mathieu/.npm-global/bin",
    "LOGNAME": "mathieu",
    "XDG_SESSION_DESKTOP": "i3",
    "QT_QPA_PLATFORMTHEME": "gtk2",
    "XDG_SESSION_TYPE": "x11",
    "XAUTHORITY": "/run/user/1000/gdm/Xauthority",
    "WINDOWPATH": "2",
    "GDM_LANG": "en_US.utf8",
    "HOME": "/home/mathieu",
    "USERNAME": "mathieu",
    "LANG": "en_GB.utf8",
    "XDG_CURRENT_DESKTOP": "i3",
    "VTE_VERSION": "5803",
    "NNN_PLUG": "t:preview-tui;v:_viu $nnn",
    "PERL5LIB": "/home/mathieu/perl5/lib/perl5",
    "NNN_COLORS": "4321",
    "XDG_SESSION_CLASS": "user",
    "TERM": "xterm-256color",
    "PERL_MB_OPT": "--install_base \"/home/mathieu/perl5\"",
    "USER": "mathieu",
    "VISUAL": "micro",
    "PERL_MM_OPT": "INSTALL_BASE=/home/mathieu/perl5",
    "DISPLAY": ":1",
    "SHLVL": "1",
    "INPUTRC": "/etc/inputrc",
    "XDG_VTNR": "2",
    "XDG_SESSION_ID": "1",
    "TILIX_ID": "258dac7a-3394-4cdd-beb0-509abf45af9b",
    "XDG_RUNTIME_DIR": "/run/user/1000",
    "QT_AUTO_SCREEN_SCALE_FACTOR": "0",
    "XDG_DATA_DIRS": "/home/mathieu/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/share:/var/lib/snapd/desktop/",
    "PERL_LOCAL_LIB_ROOT": "/home/mathieu/perl5",
    "PATH": "/home/mathieu/perl5/bin:/home/mathieu/.npm-global/bin:/home/mathieu/.local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/snap/bin",
    "HISTIGNORE": "&:[bf]g:exit",
    "GDMSESSION": "i3",
    "NNN_SSHFS": "sshfs -o allow_other,default_permission",
    "DBUS_SESSION_BUS_ADDRESS": "unix:abstract=/tmp/dbus-iCGK4xvRnm,guid=c375ebf4890c3ba9e87a7c1b5ed6a3df",
    "_": "./svg-term",
    "OLDPWD": "/home/mathieu"
  },
  "userLimits": {
    "core_file_size_blocks": {
      "soft": 0,
      "hard": 0
    },
    "data_seg_size_kbytes": {
      "soft": "unlimited",
      "hard": "unlimited"
    },
    "file_size_blocks": {
      "soft": "unlimited",
      "hard": "unlimited"
    },
    "max_locked_memory_bytes": {
      "soft": "unlimited",
      "hard": "unlimited"
    },
    "max_memory_size_kbytes": {
      "soft": "unlimited",
      "hard": "unlimited"
    },
    "open_files": {
      "soft": 524288,
      "hard": 524288
    },
    "stack_size_bytes": {
      "soft": 8388608,
      "hard": "unlimited"
    },
    "cpu_time_seconds": {
      "soft": "unlimited",
      "hard": "unlimited"
    },
    "max_user_processes": {
      "soft": 31142,
      "hard": 31142
    },
    "virtual_memory_kbytes": {
      "soft": "unlimited",
      "hard": "unlimited"
    }
  },
  "sharedObjects": [
    "linux-vdso.so.1",
    "/usr/lib64/libz.so.1",
    "/usr/lib64/libuv.so.1",
    "/usr/lib64/libnghttp2.so.14",
    "/usr/lib64/libicui18n.so.65",
    "/usr/lib64/libicuuc.so.65",
    "/usr/lib64/libdl.so.2",
    "/usr/lib64/libstdc++.so.6",
    "/usr/lib64/haswell/libm.so.6",
    "/usr/lib64/libgcc_s.so.1",
    "/usr/lib64/libpthread.so.0",
    "/usr/lib64/haswell/libc.so.6",
    "/usr/lib64/ld-linux-x86-64.so.2",
    "/usr/lib64/librt.so.1",
    "/usr/lib64/libicudata.so.65",
    "/usr/lib64/libnss_files.so.2",
    "/usr/lib64/libnss_mymachines.so.2",
    "/usr/lib64/libnss_mdns4_minimal.so.2",
    "/usr/lib64/libnss_dns.so.2",
    "/usr/lib64/libresolv.so.2"
  ]
skyzyx commented 4 years ago
export NODE_OPTIONS="--max-old-space-size={megabytes}"
skyzyx commented 4 years ago

Having said that, I'm allocating 20 GB (yes, GB) of memory for a 20-something-second asciinema file, and it's hitting a memory limit. :/

Kabouik commented 4 years ago

Oh well. I guess I'm not converting that cast to .svg then! Thanks for the clarification @skyzyx.

skyzyx commented 4 years ago

After playing with this a little more today, I'm seeing that if you set --optimize=false, then it doesn't try to run SVGO. It finishes and doesn't run out of memory for me anymore.

If you still want to optimize, you can use any number of other tools. I use ImageOptim on macOS, but there are other tools as well.

Kabouik commented 4 years ago

Excellent, thanks. It took ages with my long cast but it worked without optimization, and the final .svg file is about 6.5 MB.