search

dojo / cli-build-widget

Command for building Dojo widgets
Other
6 stars 17 forks source link

Using thread-loader to fix crash when compiling all widgets without onlyCompileBundleFiles #55

Closed rorticus closed 5 years ago

rorticus commented 5 years ago

When you turn off onlyCompileBundleFiles for ts-loader, you get an out of memory error when compiling all the widgets. You can fix this problem by compiling each widget individually but that's a bummer too as it takes a lot longer to compile.

This PR adds one fix, which is to use the threads-loader plugin to run multiple instances of ts-loader in parallel, each one having its own memory space. Note that this is very taxing on your system, but the build finishes in about 3 1/2 minutes and does not crash. Apparently we need to use happyPackMode with ts-loader when compiling with thread-loader. As a side effect, syntactic errors are no longer found by ts-loader. This is fixed by using ForkTsCheckerWebpackPlugin with the checkSyntacticErrors flag. You can see a full of example of the required configuration in the ts-loader w/ threads-loader example

Why turn turn off onlyCompileBundleFiles anyways? We might, cough cough, have a typescript transformer that needs to read types from @dojo/framework and it can't find them if onlyCompileBundleFiles is turned on.

rorticus commented 5 years ago

It's worth noting that this really messes with the error reporting...

I made an obvious semantic error and the same error gets printed like 20 times in a row,

/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.,/Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts
ERROR in /Users/rorymulligan/Desktop/Dojo2/widgets/src/accordion-pane/index.ts(27,1):
TS2304: Cannot find name 'adfadf'.

I assume maybe once per thread?

matt-gadd commented 5 years ago

@rorticus going to close this now due to: https://github.com/dojo/cli-build-widget/pull/56