Webpack fails to build superset-frontend for production build

[DarwinDennis003]

Done npm ci Node version v16.20.2 Machine : Windows 11 Tried to run npm run build

Observation : Build hangs at 94%

superset@0.0.0-dev build cross-env NODE_OPTIONS=--max_old_space_size=8192 NODE_ENV=production BABEL_ENV="${BABEL_ENV:=production}" webpack --mode=production --color

[Superset Plugin] Use symlink source for @superset-ui/chart-controls @ ./packages/superset-ui-chart-controls [Superset Plugin] Use symlink source for @superset-ui/core @ ./packages/superset-ui-core [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-calendar @ ./plugins/legacy-plugin-chart-calendar [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-chord @ ./plugins/legacy-plugin-chart-chord [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-country-map @ ./plugins/legacy-plugin-chart-country-map [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-event-flow @ ./plugins/legacy-plugin-chart-event-flow [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-heatmap @ ./plugins/legacy-plugin-chart-heatmap [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-histogram @ ./plugins/legacy-plugin-chart-histogram [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-horizon @ ./plugins/legacy-plugin-chart-horizon [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-map-box @ ./plugins/legacy-plugin-chart-map-box [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-paired-t-test @ ./plugins/legacy-plugin-chart-paired-t-test [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-parallel-coordinates @ ./plugins/legacy-plugin-chart-parallel-coordinates [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-partition @ ./plugins/legacy-plugin-chart-partition [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-rose @ ./plugins/legacy-plugin-chart-rose [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-sankey @ ./plugins/legacy-plugin-chart-sankey [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-sankey-loop @ ./plugins/legacy-plugin-chart-sankey-loop [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-sunburst @ ./plugins/legacy-plugin-chart-sunburst [Superset Plugin] Use symlink source for @superset-ui/legacy-plugin-chart-world-map @ ./plugins/legacy-plugin-chart-world-map [Superset Plugin] Use symlink source for @superset-ui/legacy-preset-chart-deckgl @ ./plugins/legacy-preset-chart-deckgl [Superset Plugin] Use symlink source for @superset-ui/legacy-preset-chart-nvd3 @ ./plugins/legacy-preset-chart-nvd3 [Superset Plugin] Use symlink source for @superset-ui/plugin-chart-echarts @ ./plugins/plugin-chart-echarts [Superset Plugin] Use symlink source for @superset-ui/plugin-chart-handlebars @ ./plugins/plugin-chart-handlebars [Superset Plugin] Use symlink source for @superset-ui/plugin-chart-pivot-table @ ./plugins/plugin-chart-pivot-table [Superset Plugin] Use symlink source for @superset-ui/plugin-chart-table @ ./plugins/plugin-chart-table [Superset Plugin] Use symlink source for @superset-ui/plugin-chart-word-cloud @ ./plugins/plugin-chart-word-cloud [Superset Plugin] Use symlink source for @superset-ui/switchboard @ ./packages/superset-ui-switchboard

22% building import loader ./node_modules/js-yaml-loader/index.js- Processing: C:\Users\trias\superset-forked\superset-frontend\packages\superset-ui-chart-c \ Processing: C:\Users\trias\superset-forked\superset-frontend\src\visualizations\TimeTable\transformProps.ts

[gnought]

I have a similar experience before. LIkely it is in the webpack stage. Suggest to have 8GB ram at least for the compliation.

[DarwinDennis003]

Thanks for the reply . Currently im running it on docker WSL .

This is my memory stats in WSL total used free shared buff/cache available Mem: 9.7Gi 642Mi 8.1Gi 3.0Mi 1.0Gi 8.8Gi Swap: 3.0Gi 59Mi 2.9Gi

Memory stats during [lean 6/10] RUN --mount=type=bind,target=./requirements/local.txt,src=./requirements/local.txt total used free shared buff/cache available Mem: 9.7Gi 2.2Gi 136Mi 4.0Mi 7.4Gi 7.3Gi Swap: 3.0Gi 62Mi 2.9Gi

Memory stats while npm run build total used free shared buff/cache available Mem: 9.7Gi 4.9Gi 177Mi 4.0Mi 4.7Gi 4.5Gi Swap: 3.0Gi 70Mi 2.9Gi

           total        used        free      shared  buff/cache   available

Mem: 9.7Gi 5.5Gi 310Mi 4.0Mi 3.9Gi 3.9Gi Swap: 3.0Gi 70Mi 2.9Gi

After it used some swap memory total used free shared buff/cache available Mem: 9.7Gi 5.8Gi 3.6Gi 3.0Mi 334Mi 3.7Gi Swap: 3.0Gi 960Mi 2.1Gi

Then it struct at this process and the memory are released => [superset-node 7/7] RUN npm run build 400.1s => => # - Processing: src/visualizations/TimeTable/index.ts
=> => # \ Processing: src/visualizations/TimeTable/transformProps.ts
=> => # | Processing: src/visualizations/dashboardComponents/DashboardComponentsRegistry.ts
=> => # / Processing: src/visualizations/dashboardComponents/ExampleComponent/ExampleComponent.tsx
=> => # - Processing: src/visualizations/dashboardComponents/ExampleComponent/index.ts => => # \ Processing: src/visualizations/presets/dashboardComponents.ts

           total        used        free      shared  buff/cache   available

Mem: 9.7Gi 4.0Gi 5.3Gi 3.0Mi 420Mi 5.4Gi Swap: 3.0Gi 868Mi 2.2Gi

And the process continues (use and release) total used free shared buff/cache available Mem: 9.7Gi 7.3Gi 1.3Gi 3.0Mi 1.1Gi 2.1Gi Swap: 3.0Gi 853Mi 2.2Gi

           total        used        free      shared  buff/cache   available

Mem: 9.7Gi 4.7Gi 3.9Gi 3.0Mi 1.1Gi 4.7Gi Swap: 3.0Gi 853Mi 2.2Gi

Time : Building 1250.8s (19/25)

Solutions or alternatives are appreciated !!

[DarwinDennis003]

When i set my heap to "build": "cross-env NODE_OPTIONS=--max_old_space_size=1120 NODE_ENV=production BABEL_ENV=\"${BABEL_ENV:=production}\" webpack --mode=production --color",

The Build failed with error message at 233178 ms as follows

234.4 [26:0x5e495e0] 233178 ms: Scavenge 876.2 (1143.9) -> 874.4 (1147.9) MB, 7.2 / 0.0 ms (average mu = 0.966, current mu = 0.648) task 234.4 [26:0x5e495e0] 233193 ms: Scavenge 880.4 (1147.9) -> 876.0 (1157.7) MB, 8.0 / 0.0 ms (average mu = 0.966, current mu = 0.648) task 234.4 [26:0x5e495e0] 233712 ms: Mark-sweep 885.6 (1157.7) -> 878.6 (1172.7) MB, 508.4 / 0.0 ms (average mu = 0.933, current mu = 0.243) allocation failure scavenge might not succeed 234.4 234.4 234.4 <--- JS stacktrace ---> 234.4 234.4 FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory 234.4 1: 0xb090e0 node::Abort() [webpack] 234.4 2: 0xa1b70e [webpack] 234.4 3: 0xce1a20 v8::Utils::ReportOOMFailure(v8::internal::Isolate, char const, bool) [webpack] 234.4 4: 0xce1dc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate, char const, bool) [webpack] 234.4 5: 0xe99435 [webpack] 234.4 6: 0xea90fd v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [webpack] 234.4 7: 0xeabdfe v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [webpack] 234.4 8: 0xe6d072 v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [webpack]
234.4 9: 0xe65684 v8::internal::FactoryBase::AllocateRawWithImmortalMap(int, v8::internal::AllocationType, v8::internal::Map, v8::internal::AllocationAlignment) [webpack] 234.4 10: 0xe676e1 v8::internal::FactoryBase::NewRawTwoByteString(int, v8::internal::AllocationType) [webpack]
234.4 11: 0xe6f19c v8::internal::Factory::NewStringFromUtf8(v8::base::Vector const&, v8::internal::AllocationType) [webpack]
234.4 12: 0xcf0615 v8::String::NewFromUtf8(v8::Isolate, char const, v8::NewStringType, int) [webpack] 234.4 13: 0xbe8e81 [webpack] 234.4 14: 0xbe8f9d node::StringDecoder::DecodeData(v8::Isolate, char const, unsigned long) [webpack] 234.4 15: 0xbe95b2 [webpack] 234.4 16: 0xd3df8e [webpack] 234.4 17: 0xd3f3af v8::internal::Builtin_HandleApiCall(int, unsigned long, v8::internal::Isolate*) [webpack] 234.4 18: 0x15d9f39 [webpack] 234.5 ✔ Lint done...

Dockerfile:48

46 | COPY ./superset-frontend ./ 47 | # This seems to be the most expensive step 48 | >>> RUN npm run ${BUILD_CMD} 49 | 50 | ######################################################################

[rusackas]

Closing this since Superset doesn't officially support windows. If anyone wants to open PRs to support scripts/packages/docs to make Windows supported, you're welcome to open PRs and SIPs to chip away at this incrementally.

[alibertz]

this exact issue is happening to me on macOS. Any ideas anybody?

[rusackas]

If you're experiencing this in Docker, it's likely that you need to throw more RAM at it. I have mine set to 16GB.

[alibertz]

It's not. I have 16GB allocated. Something definitely goes wrong toward the end of npm run build step in my docker build script, but it doesn't even give an error, it just hangs

[DarwinDennis003]

It's not. I have 16GB allocated. Something definitely goes wrong toward the end of npm run build step in my docker build script, but it doesn't even give an error, it just hangs

We could not able to find the root cause of the issue . It seems that when we zip download superset the build works fine but yes with some hang or delay but when we clone and run npm build it hangs forever on both docker and local webpack build stage.

Can you try for zip build and share the observation. Thanks

[alibertz]

Update: build completed just fine with a complete uninstall/reinstall of docker on my machine