search

sdetweil / MagicMirror_scripts

Magic Mirror installation and setup scripts
337 stars 61 forks source link

Enhancement: Abort Install and Update-Scripts if mixed (32bit vs. 64bit) systems are detected #72

Closed Tom-Hirschberger closed 8 months ago

Tom-Hirschberger commented 8 months ago

Hi Sam,

had/have a lot of problems to update to the current version of MagicMirror (2.25.0) with my production system. I tried different ways with or without your scripts but nothing worked. But after some investigation i found the reason.

I switched to 64bit Bullseye very early and did not realize that the early versions boot a 64bit kernel but use a 32bit userland. Here are some outputs of my system:

If i do the same commands on a system i flashed today the output of file differs:

With version 2.23.0 of MagicMirror and node 16.x things worked but the new electron versions of 2.24.0 and 2.25.0 do not run on the 32bit userland. If i override the arch settings of npm during installation i can install the 32bit version of electron and the mirror starts. But i can not get my GPIO module to get compiled with electron-rebuild.

I do not think there is a way to get the mixed system running with the current electron version completely.

My suggestion would be to integrate a check to the install and upgrade script if the system uses 32bit userland although the arch is aarch64 and abort the installation in this case.

One additional thing i noticed is that if i use your scripts my nodejs installation gets purged even it is the correct version already. Maybe you can add a version check for node before removing it.

I do not know if i have time to create a pull request for the scripts in the next days but i will try so. If i don't i wanted to write down my results at least.

sdetweil commented 8 months ago

someone suggested to use 

dpkg --print-architecture

to get the architecture instead of uname

it returns arm64 instead of aarch64...

can u run it there?

the latest 64 bit image pproduces

Linux raspberrypi 5.15.84-v8+ #1613 SMP PREEMPT Thu Jan 5 12:03:08 GMT 2023 aarch64 GNU/Linux

pi@raspberrypi:$ file /bin/bash
/bin/bash: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=96c22dfc3c74ccf4ba77d9cce6fc2c5e635456c1, for GNU/Linux 3.7.0, stripped
pi@raspberrypi:$ dpkg --print-architecture
arm64

so I see your point, but I could just as easily set arch to arm32 or armv7l to get the 32 bit user space node.

I've been considering changing to using n to support install and upgrade of node/npm https://www.npmjs.com/package/n/v/5.0.1

Tom-Hirschberger commented 8 months ago

On my mixed system i get the following result:

pi@mirror-eg:~ $ dpkg --print-architecture
armhf

After the initial installation of node i am a big fan of n to upgrade.

I forced the arch of npm to armv7l with export npm_config_arch=armv7l in my /etc/bash.bashrc and could install MagicMirror successfully but i could not get the PIR modules npm lib onoff to be compiled with electron-rebuild even if i set the arch of electron rebuild with the -a option to armv7l, too.

sdetweil commented 8 months ago

did you use my MMM-PIR_Sensor, which fixes the electron rebuild?

https://github.com/sdetweil/MMM-PIR-Sensor

electron-rebuild CANNOT be used from the module folder anymore

sdetweil commented 8 months ago

does n install the correct node/npm on your system?

sdetweil commented 8 months ago

can u tell me which image you used? I don't see Apr 3 anywhere in the list Screenshot at 2023-10-10 10-03-26

Tom-Hirschberger commented 8 months ago

I think the kernel changed in the meantime as i updated the system a couple of times since the first installation with apt dist-upgrade. The initial installation might be a year ago.

Edit: Found a way to determine the first start of the image:

pi@mirror-eg:~ $ sudo tune2fs -l /dev/mmcblk0p2|grep 'Filesystem created' 
Filesystem created:       Thu Sep 22 02:32:56 2022

I use my own GPIO module https://github.com/Tom-Hirschberger/MMM-GPIO-Notifications in my installation. But i switched to your postinstall script in the past as you suggested it.

Tom-Hirschberger commented 8 months ago

Sorry, missed one comment... n installs the correct version but you need an existing nodejs installation with npm installed already to use it. The systems nodejs installation will work for it i think but you then need to install nodejs and npm to let it replace of n a couple of seconds later.

sdetweil commented 8 months ago

n installs the correct version but you need an existing nodejs installation with npm installed already to use it.

yes..

I let the system pick the default node/npm.. and then install n testing

sdetweil commented 8 months ago

interesting the 09-26 entry contains files with 09-22 dates...

automation in its best form... lol...

downloading to test now

sdetweil commented 8 months ago

the 9/22 image now shows aarch64 for the file /bin/bash.. not armhf

Tom-Hirschberger commented 8 months ago

I flashed the 2022-04-04 to test and can confirm that now the Userland is 64bit in this image.

I found another way to determine the initally flashed image. This is the info of the mixed system:

cat /etc/rpi-issue 
Raspberry Pi reference 2022-09-22
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 8a42abcd1dbd8c9c1fdfca4e0c3778255b2f9cc4, stage4
sdetweil commented 8 months ago

@Tom-Hirschberger I think I have a general fix using n without having to know the platform architecture

want to try them?

note that they are in the testit branch

install 

bash -c  "$(curl -sL https://raw.githubusercontent.com/sdetweil/MagicMirror_scripts/testit/raspberry.sh)"

and upgrade

bash -c  "$(curl -sL https://raw.githubusercontent.com/sdetweil/MagicMirror_scripts/testit/upgrade-script.sh)" force

so my process is

sudo apt-get update if no node installed, sudo apt-get install nodejs -y

if no npm sudo apt-get install npm -y

if npm is installed but n is not, install n sudo npm i n -g

use n to upgrade node/npm if downlevel (default 64bit node is 12.2)

tried on both bullseye 64 bit, and one 32 bit buster legacy

Tom-Hirschberger commented 8 months ago

Great. I will try to test it tomorrow but I can not promise that I have time to

sdetweil commented 8 months ago

understood..

sdetweil commented 8 months ago

also see https://forum.magicmirror.builders/topic/18072/install-upgrade-scripts-changed-for-version-2-25-and-node-20

Tom-Hirschberger commented 8 months ago

Hi,

i tried the new upgrade script today on my "mixed" system:

pi@mirror-eg:~ $ bash -c  "$(curl -sL https://raw.githubusercontent.com/sdetweil/MagicMirror_scripts/testit/upgrade-script.sh)" force 
force: line 145: /usr/local/bin/node: No such file or directory
doing test run = false
update log will be in /home/pi/MagicMirror/installers/upgrade.log
Check current Node installation ...
Node currently installed. Checking version number.
force: line 210: /usr/local/bin/node: No such file or directory
Minimum Node version: v20.8.0
Installed Node version: V1.0.0
Node should be upgraded.
Installing Node.js ...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be upgraded:
  nodejs
1 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
Need to get 0 B/27.0 MB of archives.
After this operation, 63.7 MB of additional disk space will be used.
apt-listchanges: Reading changelogs...
(Reading database ... 113408 files and directories currently installed.)
Preparing to unpack .../nodejs_20.5.1-deb-1nodesource1_armhf.deb ...
Unpacking nodejs (20.5.1-deb-1nodesource1) over (16.20.2-deb-1nodesource1) ...
Setting up nodejs (20.5.1-deb-1nodesource1) ...
Processing triggers for man-db (2.9.4-2) ...
node version is force: line 289: /usr/local/bin/node: No such file or directory
force: line 291: /usr/local/bin/node: No such file or directory
Node.js installation Done! version=

I think it is messed up completely. As the release of Raspberry OS does not exist anymore i think i will clean up and start with a new Bullseye 64bit installation. But i will try the new installation and upgrade scripts at least for Bullseye 64bit.

Tom-Hirschberger commented 8 months ago

~Clean Bullseye 32Bit installation:~ Edit: Clean Bookworm 32Bit installation:

pi@mirror-eg:~ $ bash -c "$(curl -sL https://raw.githubusercontent.com/sdetweil/MagicMirror_scripts/testit/raspberry.sh)"

$$\      $$\                     $$\           $$\      $$\ $$\                                          $$$$$$\
$$$\    $$$ |                    \__|          $$$\    $$$ |\__|                                        $$  __$$\
$$$$\  $$$$ | $$$$$$\   $$$$$$\  $$\  $$$$$$$\ $$$$\  $$$$ |$$\  $$$$$$\   $$$$$$\   $$$$$$\   $$$$$$\  \__/  $$ |
$$\$$\$$ $$ | \____$$\ $$  __$$\ $$ |$$  _____|$$\$$\$$ $$ |$$ |$$  __$$\ $$  __$$\ $$  __$$\ $$  __$$\  $$$$$$  |
$$ \$$$  $$ | $$$$$$$ |$$ /  $$ |$$ |$$ /      $$ \$$$  $$ |$$ |$$ |  \__|$$ |  \__|$$ /  $$ |$$ |  \__|$$  ____/
$$ |\$  /$$ |$$  __$$ |$$ |  $$ |$$ |$$ |      $$ |\$  /$$ |$$ |$$ |      $$ |      $$ |  $$ |$$ |      $$ |
$$ | \_/ $$ |\$$$$$$$ |\$$$$$$$ |$$ |\$$$$$$$\ $$ | \_/ $$ |$$ |$$ |      $$ |      \$$$$$$  |$$ |      $$$$$$$$\
\__|     \__| \_______| \____$$ |\__| \_______|\__|     \__|\__|\__|      \__|       \______/ \__|      \________|
                       $$\   $$ |
                       \$$$$$$  |
                        \______/

install log being saved to /home/pi/install.log
cat: /boot/issue.txt: No such file or directory
system running in command line mode, configured for graphical desktop, please reboot
sdetweil commented 8 months ago

are you running in cli mode? or desktop mode?

Tom-Hirschberger commented 8 months ago

Im installing directly in a shell on the desktop. The system is running in graphical mode but the file /boot/issue.txt does not exist. Flashed the image 30 minutes ago with the current raspberry imager.

Edit: Did a reboot but the file is still not there. Testing Bullseye 64Bit now

sdetweil commented 8 months ago

on the mixed

 /usr/local/bin/node: No such file or directory

that usually happens when n or nvm is used to remove a loaded version, but the updated PATH is not freshened. I don't delete anything anymore

I guess I could refresh the path before I try to use node

Tom-Hirschberger commented 8 months ago

I think the problem on the mixed system is that a binary is installed which can not be run on the current architecture. The file is there but even after a reboot the file is not executable without this error.

Tom-Hirschberger commented 8 months ago

~Installation Bullseye 64bit works great~ Edit: Wrong version flashed as current stable is Bookworm now

Tom-Hirschberger commented 8 months ago

Damn. The mirror with the os images of Raspberry OS is being updated at the moment. No more Buster images anymore.

New Stable is Bookworm. Old Stable is Bullseye.

Tom-Hirschberger commented 8 months ago

I corrected my comment above. I flashed Bookworm instead of Bullseye in the Imager as the new stable is Bookworm. So Bookworm 32Bit does not contain /boot/issue.txt file

sdetweil commented 8 months ago

bookworm. sheesh.. always something...

I am busy this morning helping neighbor, will review.

thanks for your clear info

Tom-Hirschberger commented 8 months ago

What the ... I tried to flash the old-stable in 32bit which is bullseye. The result is a mixed system. 64bit kernel, 32bit binaries

Tom-Hirschberger commented 8 months ago

So...Some first conclusion:

$$\ $$\ $$\ $$\ $$\ $$\ $$$$$$\ $$$\ $$$ | _| $$$\ $$$ |_| $$ $$\ $$$$\ $$$$ | $$$$$$\ $$$$$$\ $$\ $$$$$$$\ $$$$\ $$$$ |$$\ $$$$$$\ $$$$$$\ $$$$$$\ $$$$$$\ _/ $$ | $$\$$\$$ $$ | __$$\ $$ $$\ $$ |$$ __|$$\$$\$$ $$ |$$ |$$ $$\ $$ $$\ $$ $$\ $$ $$\ $$$$$$ | $$ \$$$ $$ | $$$$$$$ |$$ / $$ |$$ |$$ / $$ \$$$ $$ |$$ |$$ | _|$$ | _|$$ / $$ |$$ | __|$$ __/ $$ |\$ /$$ |$$ $$ |$$ | $$ |$$ |$$ | $$ |\$ /$$ |$$ |$$ | $$ | $$ | $$ |$$ | $$ | $$ | _/ $$ |\$$$$$$$ |\$$$$$$$ |$$ |\$$$$$$$\ $$ | _/ $$ |$$ |$$ | $$ | \$$$$$$ |$$ | $$$$$$$$\ _| _| ___| _$$ |_| _____|_| _|_|_| _| _____/ _| ___| $$\ $$ | \$$$$$$ | \/

install log being saved to /home/pi/install.log Installing helper tools ... Updating packages ... [ <=> ] installing correct version of node and npm, please wait ERROR: npm v10.1.0 is known not to run on Node.js v12.22.12. This version of npm supports the following node versions: ^18.17.0 || >=20.5.0. You can find the latest version at https://nodejs.org/.

ERROR: /usr/local/lib/node_modules/npm/lib/utils/exit-handler.js:19 const hasLoadedNpm = npm?.config.loaded ^

SyntaxError: Unexpected token '.' at wrapSafe (internal/modules/cjs/loader.js:915:16) at Module._compile (internal/modules/cjs/loader.js:963:27) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10) at Module.load (internal/modules/cjs/loader.js:863:32) at Function.Module._load (internal/modules/cjs/loader.js:708:14) at Module.require (internal/modules/cjs/loader.js:887:19) at require (internal/modules/cjs/helpers.js:74:18) at module.exports (/usr/local/lib/node_modules/npm/lib/cli-entry.js:15:23) at module.exports (/usr/local/lib/node_modules/npm/lib/es6/validate-engines.js:39:10) at module.exports (/usr/local/lib/node_modules/npm/lib/cli.js:4:31) /usr/local/bin/n: Zeile 744: /usr/local/bin/node: Datei oder Verzeichnis nicht gefunden bash: Zeile 266: /usr/local/bin/node: Datei oder Verzeichnis nicht gefunden node failed to install, exiting


* Raspberry Bookworm 32Bit 2023-10-10: 64bit Kernel, 32bit Binaries, Installation fails cause of missing /boot/issue.txt
* Raspberry Bookworm 64Bit 2023-10-10: 64bit Kernel, 64bit Binaries, Installation works
sdetweil commented 8 months ago

super, thanks! helping neighbor this morning. posted a warning on forum.

I never install mm ON the pi, always over ssh

sdetweil commented 8 months ago

can u send me the install.log from the bullseye ,64/32 failure if u still have it. n install node worked, npm is there, node is not??.....

apt-get update. yeh I do it after instead of before.. will fix, thx

Tom-Hirschberger commented 8 months ago

On the 32bit systems even if n is used the wrong version (arm64) of node is installed. If i force the arch to armv7l by sudo n -a armv7l 20.8.0 the correct version is installed. After this a hash -r command is needed to refresh cache and use the new node version in this shell.

sdetweil commented 8 months ago

/boot/issue.txt isn't needed anymore it was just a doc thing in the log

Tom-Hirschberger commented 8 months ago

The missing node is caused of the wrong binary which can not be executed on the 32bit systems.

But i do not think it will be possible to use electron-rebuild on the 32bit systems even if the mirror gets installed correctly.

sdetweil commented 8 months ago

missing node, got it.

I use PATH=$PATH instead of hash -rf

will review

Tom-Hirschberger commented 8 months ago

Tried to use electron-rebuild. I changed my postinstall-script to:

#!/bin/bash

if [ ! -f ../../node_modules/.bin/electron-rebuild ]; then
    cd ../..
    npm install --arch=armv7l @electron/rebuild
    cd -
fi 
../../node_modules/.bin/electron-rebuild -a armv7l

But even with the forced arch it fails.

sdetweil commented 8 months ago

But even with the forced arch it fails.

what's the failure?

sdetweil commented 8 months ago

fixed the issue.txt, and moved apt-get update to the start...

Tom-Hirschberger commented 8 months ago

I tried the electron-rebuild without specifiying any arch. Then it worked without any error. So only the npm and n commands need a arch

sdetweil commented 8 months ago

i did the armf image, 2023-10-10-raspios-bookworm-armhf.img.xz

see armv7l from uname... all things work I installed mmm-pir-sensor and electron-rebuild worked as expected

pi@raspberrypi:~/MagicMirror/modules $ git clone https://github.com/sdetweil/MMM-PIR-Sensor
Cloning into 'MMM-PIR-Sensor'...
remote: Enumerating objects: 187, done.
remote: Counting objects: 100% (60/60), done.
remote: Compressing objects: 100% (39/39), done.
remote: Total 187 (delta 24), reused 47 (delta 21), pack-reused 127
Receiving objects: 100% (187/187), 65.60 KiB | 959.00 KiB/s, done.
Resolving deltas: 100% (95/95), done.
pi@raspberrypi:~/MagicMirror/modules $ cd MMM-PIR-Sensor/

pi@raspberrypi:~/MagicMirror/modules/MMM-PIR-Sensor $ npm install

> Magic-Mirror-Module-PIR-Sensor@1.1.0 postinstall
> ./postinstall

⠦ Building module: epoll, Completed: 0make: Entering directory '/home/pi/MagicMirror/modules/MMM-PIR-Sensor/node_modules/epoll/build'
  CXX(target) Release/obj.target/epoll/src/epoll.o
⠏ Building module: epoll, Completed: 0  SOLINK_MODULE(target) Release/obj.target/epoll.node
⠦ Building module: epoll, Completed: 0  COPY Release/epoll.node
make: Leaving directory '/home/pi/MagicMirror/modules/MMM-PIR-Sensor/node_modules/epoll/build'
✔ Rebuild Complete

added 6 packages, and audited 7 packages in 7m

found 0 vulnerabilities
sdetweil commented 8 months ago

So only the npm and n commands need a arch

I didn't add arch to any command.. I updated testit branch with all the latest updates.. note I had an error when comparing for bullseye (on bookworm) that caused a bad , non n , node install...

that is fixed

sdetweil commented 8 months ago

also moved screensaver setup in front of pm2 setup

Tom-Hirschberger commented 8 months ago

Thank you for your quick response and affort. I will try as fast as I can but it may take some time as I am on a trip the next days and do not have access to my mirrors.

sdetweil commented 8 months ago

Understood... I just used the new 64 bit image and it worked correctly too.. no arch.. both sides aarch64

I appreciate all your investigation and suggestions..

sdetweil commented 8 months ago

given your info, I was able to fix install and upgrade to handle this mixed environment.

sdetweil commented 8 months ago

@Tom-Hirschberger have u had any time to test, what do you think?

Tom-Hirschberger commented 8 months ago

Hi and sorry for the late response. I am ill at the moment and need some more time to test. But i suggest the script to two persons in the last days and only got positive feedback 😉 I had problems with pm2 not automatically starting and the new vnc server on Bookworm but could not test further yet.

sdetweil commented 8 months ago

thanks.. I hope you get well soon.. no rush, just wanted to communicate

pm2 startup, yes there is a bug in pm2.. working on it now well, I found the problem with the scripts and pm2 boot startup and fixed my scripts..

Tom-Hirschberger commented 8 months ago

Hi, tried to upgrade today but without luck... upgrade.log

Tom-Hirschberger commented 8 months ago

Will try a to remove Nodejs and MagicMirror now and do a "clean" install

Tom-Hirschberger commented 8 months ago

Does not work either... install.log

Tom-Hirschberger commented 8 months ago

I needed to force the npm arch to armhf again to install the correct node version (export npm_config_arch=armhf) before installation. Then i was able to install the correct version of node. After this i needed to set "npm_config_arch" to blank to get the correct electron_rebuild version to be installed by the GPIO module. It's still a mess. I will switch to 64bit Bullseye as fast as i can.