tdurand
commented
3 years ago Some trial and error progress notes:
Found out this project while looking into the "webpack" of node.js : https://github.com/vercel/ncc , it is from the same people doing next.js
It is supposed to be able to build your node.js app into a single index.js file (which is huge) containing all the node_modules dependencies you need.. then you just need to do: node index.js without needing the node_module folder.
This seems very much what I'm looking to do.. and I was able to make it work properly on a very simple example project that uses express: https://github.com/vercel/next.js/tree/canary/examples/custom-server .
Final app necessary files:
What you need to do is:
# create the .next folder containing the client side code
npm i
npm run build
# pack the node.js server side code in the dist/ folder
ncc build server.js
# this creates a dist/ folder with an index.js in it (18 MB with express.js)
# then you need to copy the client side code to the dist/ folder (so the index.js app can use it)
cp -r .next/ dist/
# then launching the app should work
cd dist/
node index.jsAlso very important you need to disable the auto routing of next.js by hooking to next to node.js this way in your server.js file
const app = next({ useFileSystemPublicRoutes: false })But while trying on OpenDataCam I run into a couple of issues:
-
On the development branch some code using the new class syntax with "private field" is failing when packing with
ncc(this is not a big deal to change, just two files) , it is a known issue of the tool: https://github.com/vercel/ncc/issues/499 -
When fixing the issue from above (using master branch for now) then I run into a new error which is super cryptic
Error: Invalid hook call. Hooks can only be called inside of the body of a function component. This could happen for one of the following reasons:
1. You might have mismatching versions of React and the renderer (such as React DOM)
2. You might be breaking the Rules of Hooks
3. You might have more than one copy of React in the same app
See https://fb.me/react-invalid-hook-call for tips about how to debug and fix this problem.
at resolveDispatcher (/mnt/c/Users/tibod/Documents/GitHub/opendatacam/node_modules/react/cjs/react.development.js:1465:13)
at useMemo (/mnt/c/Users/tibod/Documents/GitHub/opendatacam/node_modules/react/cjs/react.development.js:1520:20)
at Provider (/mnt/c/Users/tibod/Documents/GitHub/opendatacam/node_modules/react-redux/lib/components/Provider.js:22:41)
at processChild (/mnt/c/Users/tibod/Documents/GitHub/opendatacam/dist/index.js:470359:14)
at resolve (/mnt/c/Users/tibod/Documents/GitHub/opendatacam/dist/index.js:470276:5)Couldn't figure out how to fix this yet... related to ncc packing several copy of React I think in the big index.js ...
Also investigating I've seen someone trying the same thing but failing also (not for the same reason: https://github.com/vercel/next.js/issues/8574 )
I will pause investigation for now as the priority is to get a beta out.. but this ncc tool is promising.... Also I think it might be a good time to get rid of express.js if we move forward with this.. as there are lightweights alternatives now..
Problem
In order to use nodejs-mobile , you need to first copy the node.js app files to the phone filesystem, you can't start node.js directly with the APK file. (todo when the App in first installed and anytime that the node.js app is changed (each app update) )
Right now we were using the "recommended" method from nodejs-mobile, which is to ship the app + node_module folder in the assets directory and recursively copy this to the phone file system https://code.janeasystems.com/nodejs-mobile/android/node-project-folder#copy-the-nodejs-project-at-runtime-and-start-from-there .
Problem with this is that with a big
node_modulefolder with quite a lot of dependency like OpenDataCam , this means copy pasting24,723 Files, 4,709 Foldersand142 MB.. And on top of adding 140 MB to the app size, copying lots of small files is slow."Out of the box" , the first app install takes 2min30s (and each update).
Two things we are looking to improve here:
copy files faster
Turns out that there was a low hanging fruit here to improve this, instead of copy pasting all the files recursively which is slow, we create a zip at build time of the node.js app and then unzip it on the phone. ( idea from stackoverflow: https://stackoverflow.com/a/42415755 )
I've tried that : https://github.com/opendatacam/opendatacam-mobile/commit/c1fcca46665b275ba9bc64fd059edf299c12c219 and we get from 2min30s to around 20 seconds .. So this is already a huge win š
Only downside is that we need to integrate that zipping on the build process. for now I'm doing it manually with this command:
Have less files (and less MBs) to copy
Thinking of having
24,723 Files, 4,709 Foldersfor142Mbof files in the node_modules folder is a bit crazy.. when the client side part of the app (actually loaded in the browser) is max 5 MB .. the server side part has obviously node.js has a dependency but not even included in those 142 MB ( it is 142 MB + the 30-50 MB of node.js) ... My guess is that it might also be only 5 - 20 MB in the end...Normally we never care about this because we do the
npm installon the server and the size of the node_modules folder doesn't matter.. what matters is how much you send to the client ( see explanation here: https://github.com/vercel/next.js/discussions/14339 ) . Here we want to also reduce this to avoid.One tool to our service for this is
npm prune --production, which you can run afternpm run buildand delete all thenode_modulesdependencies that are listed as "devDependencies" in the package.json: https://github.com/opendatacam/opendatacam/blob/master/package.json#L43Out of the box doing this already reduce a bit, down to
115 MB (121,226,417 bytes) and 22,298 Files, 4,231 FoldersWhat I need to do next is to improve this, by:
npm prune(which is I think just copy/do not copy) , here is an analyse of the dependencies tree .. for example ournode-moving-things-trackeris 10 MB because it ships MOT17 benchmark files... and the node.js is really using 3 javascript files, maybe 100 kB max š