robertsLando
commented
11 months ago Hi David. Thanks for your issue, I will try to answer to all your questions.
Are we doing it wrong? Are there any parts which can be left out? Here are some of the largest files....
Everything inside node_modules folder is needed. Apart from that I use this command after building ui and server to cleanup useless files:
find . -mindepth 1 -maxdepth 1 \
! -name "node_modules" \
! -name "snippets" \
! -name ".git" \
! -name "package.json" \
! -name "server" \
! -name "dist" \It essentially keeps only the folders/files that are listed and remove the rest. In-fact if you check what's inside docker you will see:
➜ ~ docker run --rm -it zwavejs/zwave-js-ui:latest /bin/sh
/usr/src/app # ls
dist node_modules package.json server snippetsIn case of mqttjs (I'm also the maintainer of that library) I can easily use esbuild in order to create a bundle with all the files needed, that's not really easy to do with zwave-js-ui as it also have a webserver and many other things that makes that hard to bundle everything in a single file, paths to things will just be wrong everywhere or less predictable.
About compiling it to bytecode, I already do that with pkg, you see the available builds in releases: https://github.com/zwave-js/zwave-js-ui/releases/tag/v9.5.1
Its virtual memory size is 180MiB. Is there anything which can be done to reduce that? Is the full UI loaded into the interpreter even when only the ZWave←→MQTT bridge is active? Is it possible to run a version with the UI completely removed? It would actually be OK even if the user has to stop the zwave-js-ui process, and restart it with UI enabled.
Everything is loaded by default, UI, webserver etc... What could help here could be to use env vars to disable the webserver, dunno how much memory that could save BTW...
backups/: Flash file system, not preserved.
Not ideal but I let you decide
config-db/: Our packaging seems to point $ZWAVEJS_EXTERNAL_CONFIG to this, but I think that's wrong because it makes another copy of everything that's already shipped in the package? We should leave this to be used "sparingly, when custom files are absolutely necessary" as the documentation says. And then it should be in the flash file system and preserved over system upgrades.
The $ZWAVEJS_EXTERNAL_CONFIG env var is used because by default when doing a configuration db update that would use the location of the actual db in node_modules and that would be overriden by an update, also in docker-env that folder is not preserverd in volumes so we set it by default on a folder in store that is hidden to the user in store.
xxxxxxxx.json, xxxxxxxx.metadata.json, xxxxxxxx.values.json: Flash file system, perhaps not preserved? They can all be recreated, I believe?
I think yes? @AlCalzone can tell you that
config/, sessions/, snippets/: Not sure, these are empty for me.
- Config is used to load users custom configuration files, dunno if you need that
- Sessions contains the users sessions, them could be stored on ram
- Snippets: users custom snippets to use with driver function, dunno if you need that
dwmw2
kpine
nxhack
AlCalzone
We package Domoticz for the OpenWrt distribution, and have historically used OpenZWave with it. Domoticz is now switching to ZWave-JS-UI so it would be great to package ZWave-JS-UI to work out of the box on OpenWrt too: https://github.com/nxhack/openwrt-node-packages/pull/1826
An always-on embedded router/AP is a perfect place to run home automation, but the environment can be a little constrained in CPU, memory and "disk" storage. For the last few years mine has been running on a MIPS24k with 128MiB of RAM.
We have a proof of concept basically working (not on that box, but on a quad Arm7 with 256MiB of RAM). I'd appreciate some help cleaning it up though, please. There are a few things to look at. Please let me know if I should file separate issues for each.
1. The package is huge — about 100MiB.
Are we doing it wrong? Are there any parts which can be left out? Here are some of the largest files....
Some of those seem like amalgamated or preprocessed versions of the original sources; in some cases we end up with what looks like three copies of the same code. Do we need to ship them all? In the case of MQTT there's a separate package for that anyway, which has the same three files that the zwave-js-ui package is shipping too.
Some help understanding how this stuff works would be much appreciated.
Also, is it possible to compile down to bytecode or even native code, and ship only that to the target system? That might help with...
2. Memory usage
Its virtual memory size is 180MiB. Is there anything which can be done to reduce that? Is the full UI loaded into the interpreter even when only the ZWave←→MQTT bridge is active? Is it possible to run a version with the UI completely removed? It would actually be OK even if the user has to stop the
zwave-js-uiprocess, and restart it with UI enabled.Of course it would be nice to have a separate minimal server, and run the UI on demand (and as much as possible on the browser side), but that's probably a lot more work. Anything we can do to reduce memory usage in a relatively non-intrusive fashion would be useful though.
3. File system usage in
/etc/zwave-js-ui.In OpenWrt we typically use a writable overlay flash file system on top of a read-only root, and have the following categories of storage:
• The initial root filesystem (including the 100MiB package, as discussed above). • Files written to flash in the overlay, which are preserved across upgrade. • Files written to flash in the overlay, which are lost across upgrade. • Files in RAM, lost on every reboot.
We try to reduce the amount written to the flash file system — both in terms of size, and frequency of writes. We also try to minimize the amount that's preserved across system upgrade, as it's packaged into a smallish tarball which is then extracted into the new system after upgrade. Looking through the contents of /etc/zwave-js-ui, here's my idea of what might go where; please let me know if it's sane...
settings.json,users.json,nodes.json,scenes.json: Preserved over sysupgrade.logs/: In RAM. Or perhaps even sent only to syslog and not to files at all.backups/: Flash file system, not preserved.config-db/: Our packaging seems to point$ZWAVEJS_EXTERNAL_CONFIGto this, but I think that's wrong because it makes another copy of everything that's already shipped in the package? We should leave this to be used "sparingly, when custom files are absolutely necessary" as the documentation says. And then it should be in the flash file system and preserved over system upgrades.*.lock: In RAMxxxxxxxx.json,xxxxxxxx.metadata.json,xxxxxxxx.values.json: Flash file system, perhaps not preserved? They can all be recreated, I believe?config/,sessions/,snippets/: Not sure, these are empty for me.