OutOfMemory crash

[SuhairZain]

Hi, While running with our project, the process runs for about 2 minutes and then crashes. The memory consumption of the process goes upto 1.44GB before it crashes. Here's the crash log if it'd help. This is a closed-source project and thus unfortunately I'm unable to share the code.

ts-unused-exports tsconfig.json src/app.tsx

<--- Last few GCs --->

[92153:0x103001400]    78640 ms: Mark-sweep 1413.2 (1462.7) -> 1413.5 (1446.7) MB, 6226.3 / 0.0 ms  (+ 0.0 ms in 0 steps since start ofmarking, biggest step 0.0 ms, walltime since start of marking 6226 ms) last resort
[92153:0x103001400]    85136 ms: Mark-sweep 1413.5 (1446.7) -> 1413.6 (1446.7) MB, 6495.7 / 0.0 ms  last resort

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0xf06f099cef1 <JSObject>
    1: resolve [path.js:~1159] [pc=0x15b7e14c13b7](this=0x17d8fd8b5491 <Object map = 0x1978d8bb7c59>)
    2: arguments adaptor frame: 2->0
    3: /* anonymous */(aka /* anonymous */) [/Users/suhairzain/.config/yarn/global/node_modules/ts-unused-exports/lib/parser.js:~153] [pc=0x15b7e14d78fc](this=0xf06f0982241 <undefined>,path=0x3059ab9e80d1 <String[4]: src/>)
    4: arguments adaptor frame: 3->1...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/Users/suhairzain/.nvm/versions/node/v8.4.0/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/Users/suhairzain/.nvm/versions/node/v8.4.0/bin/node]
 3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/Users/suhairzain/.nvm/versions/node/v8.4.0/bin/node]
 4: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [/Users/suhairzain/.nvm/versions/node/v8.4.0/bin/node]
 5: v8::internal::LoadIC::LoadFromPrototype(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Smi>) [/Users/suhairzain/.nvm/versions/node/v8.4.0/bin/node]
 6: v8::internal::IC::ComputeHandler(v8::internal::LookupIterator*) [/Users/suhairzain/.nvm/versions/node/v8.4.0/bin/node]
 7: v8::internal::LoadIC::UpdateCaches(v8::internal::LookupIterator*) [/Users/suhairzain/.nvm/versions/node/v8.4.0/bin/node]
 8: v8::internal::LoadIC::Load(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Name>) [/Users/suhairzain/.nvm/versions/node/v8.4.0/bin/node]
 9: v8::internal::Runtime_LoadIC_Miss(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/suhairzain/.nvm/versions/node/v8.4.0/bin/node]
10: 0x15b7e13840dd
11: 0x15b7e14c13b7
[1]    92153 abort      ts-unused-exports tsconfig.json src/app.tsx

[pzavolinsky]

Hi @SuhairZain since this is a closed-source project and you can't share a full repro, I'll need your help tracking down the bug on your end.

I suspect problem might come from a dependency loop that ts-unused-exports does not detect, and thus gets stuck fetching and loading the same files over and over until it runs out of memory.

Could you console.log({ path, otherFiles: otherFiles.map(f => f.path) }) before this line ./node_modules/ts-unused-exports/lib/parser.js:182 and check the output to see if you find some strange pattern (i.e. a path that appears in both arrays, repeating calls with the exact same values for both arrays, etc.)

Maybe you could share just that output. Note that that will disclose filenames but not file content (do check with your team if that is OK for your project).

Just out of curiosity, could you share some stats from your project like:

• Number of TS files: find YOUR_SRC_DIR -path '*.tsx' -o -path '*.ts' | wc -l
• Number of imports: find YOUR_SRC_DIR -path '*.tsx' -o -path '*.ts' | xargs cat | grep '^import.*\.' | wc -l
• Number of exports: find YOUR_SRC_DIR -path '*.tsx' -o -path '*.ts' | xargs cat | grep '^export ' | wc -l

[SuhairZain]

Hi @pzavolinsky Following are the stats that you asked for: Number of TS files: 144 Number of exports: 436

For the number of imports, the command you provided (^import.*\.) gave a very low number, so I tried another regex: ^import.* from .* and this seems to give a number more in line with what I'd expect and also very close to what a search in VSCode gave: 924.

[SuhairZain]

I added the changes you mentioned and the output log is 105MB in size. I'll go through it to see if I can find something. In the meantime, I'll also consult with my team if it's okay to share the logs. What I did find interesting is that towards the end, there are multiple files whose otherFiles come to 30k+ in number. I think that such a large number is indeed a cyclic dependency. I'll run some other tools to see if there are any such and let you know.

[pzavolinsky]

Hi @SuhairZain I'm closing this issue. If you are still facing these problems and can provide more info so that we can troubleshoot it together fell free to reopen it.