[tech] Reduce bundle size for _app file

[Atmosfearful]

For the site repo (I haven't looked at app yet) the Nextjs _app bundle is growing quite dramatically. I have a few suspects in mind:

• lib/components is not properly tree shaking?
• every abi json is being included?
• some libraries we've added?

+ First Load JS shared by all                      1.01 MB
  ├ chunks/framework-81942bc1e06a8d22.js           42.1 kB
  ├ chunks/main-c76117333b9674d3.js                26.6 kB
  ├ chunks/pages/_app-17a9b104f534882a.js          937 kB  // 😵💀💀💀
  ├ chunks/webpack-d5ab764da11c7a9c.js             2.29 kB
  └ css/9ac0ea7000e013d4.css                       1.54 kB

[biwano]

@Atmosfearful @ladytrekker Hey, do we have some kind of history of builds. This way the culprit(s) could be easily found.

[Atmosfearful]

@biwano you can try git bisect for old commits and running npm run build-site locally and observe the console output

You can look through the vercel build logs as well, clicking the green checkmarks. But I will have to give you vercel access first

We also have this for the app but need to install it for site still https://www.npmjs.com/package/@next/bundle-analyzer

[biwano]

@Atmosfearful By bisecting I found two principal culprits.

• 58eafeb7ca6d4184f4acdcf7470b83e98da05c7f (5 May) made the js bundle go from 142Kb to 643Kb. It is a big squashed commit for pledge. It could help if someone still had the original commit history but I think that by investigating the additions to package.json I can find what happened.

• 4e89a5cfe6dacf724521f7f9228faa81726ad68c (21 June) made the js bundle go from 728Kb to 986Kb. This one is also a big squashed commit for the new web3 provider. Maybe one of the providers has too many export... This commit impacted both app and site

I keep diving.

[0xAeterno]

@biwano bit of context for you. https://github.com/KlimaDAO/klimadao/commit/58eafeb7ca6d4184f4acdcf7470b83e98da05c7f introduces web3modal on site. That package itself is super heavy 300-400kb from memory. I sussed the bundles a while back when adding new libraries, ethers and web3modal make up the majority of the bundle size 😱

[biwano]

Clue for crpto-browserify: https://github.com/vercel/next.js/discussions/13631

[biwano]

Some research notes:

• Created a hollow version of the app (1 page, no translations) : First load JS = 834kB
• Removed dependency on Torus => First load JS = 547kB (-287Kb) ~ Removes crypto browserify bundle
• Removed dependency on WalletConnect => First load JS = 314Kb (-233Kb)
• Removed dependency on Web3Modal => First load JS = 104Kb (-210Kb)

Some references:

• Issues with bundle sizes: https://github.com/web3/web3.js/issues/3267
• Web3Modal alternative: https://docs.blocknative.com/onboard (Only one I found with Torus, WalletConnect & Metamask support)

[biwano]

Output of duplicate-package-checker-webpack-plugin

  Multiple versions of @babel/runtime found:
    7.15.4 /home/biwano/code/klimadao/~/next/dist/compiled/@babel/runtime
    7.19.0 /home/biwano/code/klimadao/~/@babel/runtime

@emotion/memoize
  Multiple versions of @emotion/memoize found:
    0.7.5 /home/biwano/code/klimadao/~/@emotion/memoize
    0.8.0 /home/biwano/code/klimadao/~/@emotion/cache/~/@emotion/memoize

@emotion/weak-memoize
  Multiple versions of @emotion/weak-memoize found:
    0.2.5 /home/biwano/code/klimadao/~/@emotion/weak-memoize
    0.3.0 /home/biwano/code/klimadao/~/@emotion/cache/~/@emotion/weak-memoize

bn.js
  Multiple versions of bn.js found:
    4.11.6 /home/biwano/code/klimadao/~/ethjs-unit/~/bn.js
    4.11.8 /home/biwano/code/klimadao/~/bn.js
    4.12.0 /home/biwano/code/klimadao/~/elliptic/~/bn.js
    5.2.0 /home/biwano/code/klimadao/~/rlp/~/bn.js
    5.2.1 /home/biwano/code/klimadao/~/@ethersproject/bignumber/~/bn.js

buffer
  Multiple versions of buffer found:
    5.7.1 /home/biwano/code/klimadao/~/buffer
    6.0.3 /home/biwano/code/klimadao/~/@coinbase/wallet-sdk/~/buffer

eth-rpc-errors
  Multiple versions of eth-rpc-errors found:
    4.0.2 /home/biwano/code/klimadao/~/@coinbase/wallet-sdk/~/eth-rpc-errors
    4.0.3 /home/biwano/code/klimadao/~/@toruslabs/torus-embed/~/eth-rpc-errors

ethereumjs-util
  Multiple versions of ethereumjs-util found:
    5.2.1 /home/biwano/code/klimadao/~/ethereumjs-util
    6.2.1 /home/biwano/code/klimadao/~/ethereumjs-abi/~/ethereumjs-util
    7.1.5 /home/biwano/code/klimadao/~/web3-utils/~/ethereumjs-util

eventemitter3
  Multiple versions of eventemitter3 found:
    4.0.4 /home/biwano/code/klimadao/~/web3-core-promievent/~/eventemitter3
    4.0.7 /home/biwano/code/klimadao/~/eventemitter3

isarray
  Multiple versions of isarray found:
    1.0.0 /home/biwano/code/klimadao/~/through2/~/isarray
    2.0.5 /home/biwano/code/klimadao/~/isarray

json-rpc-engine
  Multiple versions of json-rpc-engine found:
    5.4.0 /home/biwano/code/klimadao/~/eth-json-rpc-middleware/~/json-rpc-engine
    6.1.0 /home/biwano/code/klimadao/~/json-rpc-engine

pify
  Multiple versions of pify found:
    3.0.0 /home/biwano/code/klimadao/~/eth-block-tracker/~/pify
    5.0.0 /home/biwano/code/klimadao/~/eth-json-rpc-filters/~/pify

preact
  Multiple versions of preact found:
    10.11.0 /home/biwano/code/klimadao/~/@coinbase/wallet-sdk/~/preact
    10.4.1 /home/biwano/code/klimadao/~/preact

react-is
  Multiple versions of react-is found:
    16.13.1 /home/biwano/code/klimadao/~/hoist-non-react-statics/~/react-is
    17.0.2 /home/biwano/code/klimadao/~/react-is

readable-stream
  Multiple versions of readable-stream found:
    2.3.7 /home/biwano/code/klimadao/~/through2/~/readable-stream
    3.6.0 /home/biwano/code/klimadao/~/readable-stream

safe-buffer
  Multiple versions of safe-buffer found:
    5.1.2 /home/biwano/code/klimadao/~/safe-buffer
    5.2.1 /home/biwano/code/klimadao/~/hash-base/~/safe-buffer

string_decoder
  Multiple versions of string_decoder found:
    1.1.1 /home/biwano/code/klimadao/~/through2/~/string_decoder
    1.3.0 /home/biwano/code/klimadao/~/string_decoder

tslib
  Multiple versions of tslib found:
    1.14.1 /home/biwano/code/klimadao/~/rxjs/~/tslib
    2.3.1 /home/biwano/code/klimadao/~/tslib

ethereumjs-util exists in 3 versions. This package is 273.7Kb https://bundlephobia.com/package/ethereumjs-util@7.1.5

[biwano]

Hi @Atmosfearful and @0xAeterno,

My understanding is that most of the bundle size is due to the Web3Modal (and providers) and their dependency on web3.js which is big and not easily tree shackable. Thus:

• Trying to optimize other parts of the code could work but the gains would be small.
• We could try to at least deduplicate the different versions of the same package (like ethereumjs-util) by using webpack aliases. But it seems ugly and risky.
• We could work on using an alternative like blocknative onboard which is based on ethers.js and should have a lighter footprint. But it will require surely a lot of work and testing.

What do you think?

[Atmosfearful]

We have a workstream ongoing for #660 which will remove our dependency on web3modal. Let's take a look at bundle sizing when that PR is ready

[Atmosfearful]

Some research notes: Created a hollow version of the app (1 page, no translations) : First load JS = 834kB Removed dependency on Torus => First load JS = 547kB (-287Kb) ~ Removes crypto browserify bundle Removed dependency on WalletConnect => First load JS = 314Kb (-233Kb) Removed dependency on Web3Modal => First load JS = 104Kb (-210Kb)

Very helpful thank you, unfortunately we need to keep torus and walletconnect but at least we can remove 200kb from web3modal