[SOLVED] Unable to install johnny-five on Raspberry 3

[maertz]

I've grabed the ISO from the BENJA page, flashed my SD-Card and booted my RPI with it. All good - works like a charm so far.

Now this fancy video from Andrea does a cool show off with johnny-five to toggle a LED, simply replace the index.html within the BENJA app, restart and you are ready to rock. Unfortunately not in my case, because johnny-five wasn't present and were not able to install. Andrea recommended to update the OS, as following:

sudo pacman -Syuw
sudo rm /etc/ssl/certs/ca-certificates.crt
sudo pacman -Su

then reinstalled/updated the dev. tools (all were up to date)

sudo pacman -S --needed --noconfirm base-devel linux-headers git

install johnny-five

sudo npm install -g johnny-five

unfortunately the issue is the same as mentioned in #7

Here is the final process, from initial start, update os to installing johnny-five full output.

[WebReflection]

I'm afraid I won't be able to test for a month 'cause I'm abroad.

Things that could be wrong maybe can be fixed looking for issues with johnny five or RPi cause from archlinux it looks that everything is there so there's nothing I can do.

[maertz]

Hi there, I hope you had a great time abroad. I've noticed that you've updated the ISO, thank you very much for that! 👍 Unfortunately it did not work out of the box but I made some progress with following steps. For some reason, I don't understand, I need to execute the stuff as "root" to make it work, in your example you simply run it through your index.html - is there some magic behind?

1) Full system update ref

Simply update the system to keep everything up to date

sudo pacman -Syuw
sudo rm /etc/ssl/certs/ca-certificates.crt
sudo pacman -Su

2) Install PIGPIO ref

Actually not sure if that step is necessary but I did it because I received few errors trying to install raspi-io without the -g flag (see next step)

rm pigpio.tar
sudo rm -rf PIGPIO
wget abyz.co.uk/rpi/pigpio/pigpio.tar
tar xf pigpio.tar
cd PIGPIO
make
sudo make install

3) Install NPM packages ref

Ensure to install them globally -g otherwise you'll receive an EPERM operation not permitted (looks like it's related to the FAT32 system but I've actually no clue "-g" just worked).

npm install -g johnny-five raspi-io

4) Test the stuff ref

Create a file name it test.js and execute it with node as "root". In case you receive an error, take a look at step 5 and redo this one.

var five = require("johnny-five");
var Raspi = require("raspi-io");
var board = new five.Board({
  io: new Raspi()
});

board.on("ready", function() {
  var led = new five.Led("P1-13");
  led.blink();
});

sudo node test.js

5) Create a symlink ref

If you received following error try to create a symlink or if that doesn't work either do it the hard way and copy the library (see ref.) Error: libpigpio.so: cannot open shared object file: No such file or directory

sudo ln -s /usr/local/lib/libpigpio.so /usr/lib/libpigpio.so
sudo ldconfig

After that, repeat step 4 and hopefully it will work.

[WebReflection]

have you tried installing npm modules via npm install --no-bin-links ?

[maertz]

Nope, just tried it and it seems to solve the issue installing the packages with -g (maybe even the symlink thingy) but still needs to be executed as root.

[WebReflection]

Benja starts npm with root privileges so I'm not sure what's the issue?

[maertz]

Alright, looks like I found the issue (sorry for the bad picture, I don't know how to take screenshots yet ^^). Not sure but it looks like the node version used in npm is newer than the one within electron? Maybe the difference comes from my system update in the beginning?

btw. I tried already npm install or rebuild but both didn't work. Also tried with the npm packages installed with "-g" and as well "locally" within the benja app - all end in the same way.

[maertz]

Alright, spending a whole day trying to solve this makes me stand at the exact same position as yesterday but at least I know what's happening.

If you install the npm packages it uses the system node version which mismatches the one within electron. So pigpio gets compiled with Node 8 but electron uses another. So I tried to simply replace the system node version with NVM and recompile the stuff to make it work. Unfortunately the NODE_MODULE_VERSION does not match in any combination of nodejs and electron.

electron releases (had to test each to get the used node module version)

electron version -> used node_module_version 1.7.5 -> 57 1.6.1 -> 53 1.4.16 -> 50

nodejs releases (they show at least the used versions)

Here the steps I've done...

Swap the electron version:

sudo npm install electron@<version> -g --unsafe-perm=true

Replace nodejs version ref

nvm install <version>
nvm use <version>

Recompile pigpio with the current system node version

cd /home/benja/.npm-packages/node_modules/raspi-io/node_modules/pigpio
node-gyp rebuild

From my understanding there are only two options to make it work: 1.) Try to build/compile electron with a specific node version where the node_module_version matches with the one of the system 2.) Try to build the pigpio package with electrons node version

I have no clue how to solve it, maybe it might be better to use pigpio in python and try to trigger/execute the stuff through node, instead to make it all work in node itself.

[WebReflection]

Apologies I have other priorities right now, but I'll look into it when I can.

Unsustainable OSS is tough ☹️

[maertz]

Sure no worries, I'll find a way, anyways thank you very much for your support! Ill keep this thread active until I found a way to solve it :)

[WebReflection]

First of all, I had to put 50pences on the Broadcom CPU 'cause it was warming up a lot.

Update the system

yaourt -Syua --noconfirm

Install nvm

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.2/install.sh | bash
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"

# now install node 0.7.4
nvm install 7.4.0
npm config delete prefix
nvm use 7.4.0

Update electron

npm install -g --no-bin-links electron@latest

Install pigpio & the rest

yaourt -S --needed --noconfirm python-raspberry-gpio
yaourt -S --needed --noconfirm python-pigpio-git
npm cache clear --force
npm install -g --no-bin-links raspi-io
npm install -g --no-bin-links johnny-five

# copy libpigpio under /usr/lib
sudo cp /usr/local/lib/libpigpio* /usr/lib/

Back to node System & prefix

nvm use system
npm config set prefix '~/.npm-packages'

Bootstrap demo page I've used on the video is here: https://gist.github.com/WebReflection/59b5b136a4599d69df736384b4fafd25

Be sure you load it avoiding the cache, edit ~/app/index.js to load fresh via .loadURL('http://localhost:8080/?_=' + Math.random());

Reboot ... did it work?

[WebReflection]

TL;DR there is a problem with node in electron and node in ArchLinux, which is always latest. This post theoretically explains how to solve: https://stackoverflow.com/questions/42616008/node-module-version-conflict-when-installing-modules-for-electron

There is no arm downgrade repository for node js so if above procedure does not work we need to wait Electron prebuilt to upgrade its internal NodeJS version.

Apologies for the inconvenient

[maertz]

electron-rebuild was the key! As assumed previously the npm packages were installed against the system node version, even if we simply switched the system node version to 7.4.0 which matches with the one used in electron (1.6.11) the node_module_version still mismatches and therefore the GPIO stuff failed within electron (for some reason).

Anyways, I've grabbed a fresh ISO to start over just to be sure there was no coincidence with this discovery ^^

Install electron-rebuild

npm install -g --save-dev electron-rebuild

Install PIGPIO and copy it's lib file to the user folder

yaourt -S --needed --noconfirm python-pigpio-git
sudo cp /usr/local/lib/libpigpio* /usr/lib/

Install johnny-five and raspi-io into the app/project folder

npm install --no-bin-links johnny-five raspi-io

Rebuild the npm packages with electron-rebuild (important part)

cd app/
electron-rebuild -v=1.6.11

That's it, probably other packages work just fine but especially with those which needs to be compiled they need to be rebuild through electron-rebuild.

update/hint Vice versa if you spawn a node process from electron which uses the system node version, ensure you are using the npm packages from the system otherwise you'll hit the same issue from the other side.

[WebReflection]

Awesome follow up, thank you so much for helping Benja out here.

Alternative

Because of this mess with Electron, builds, unmatched OS VS Electron node version, I'm working on an alternative fully based on WebKitGTK+

I'm literally building the ISO now but I have already prepared the necessary software to make it happen.

Goals:

• there is no Electron (so the project would lose its E in the name?)
• the running nodejs is exactly the OS one. Every package compatible with it, will just work
• since there is no Electron, how are users supposed to interact with NodeJS from the browser?

The solution is based on Web technologies such Web Sockets and Web Worker.

Meet node-worker, a module born yesterday evening.

If your ./app has a workers folder and such folder contain this utils.js file

// listen to any message such
// {name: 'ipv4'} or
// {name: 'utility', arguments: [1, 2, 3]}
// postMessage the result as soon as it's available
onmessage = (e) => {
  const name = e.data ? (e.data.name || '') : '';
  if (utils.hasOwnProperty(name)) {
    // even if synchronous, use Promises
    // so no matter if the utility is sync or async
    // since the onmessage => postMessage operation
    // is always asynchronous anyway
    Promise.resolve(
      utils[name].apply(null, e.data.arguments || [])
    ).then(postMessage);
  } else {
    postMessage('Unknown utility: ' + name);
  }
};

// exposed utilities (just one for demo purpose)
const utils = {
  ipv4() {
    const ni = require('os').networkInterfaces();
    return Object.keys(ni).reduce((out, key) => {
      return ni[key].reduce((out, iface) => {
        if (
          iface.family === 'IPv4' &&
          iface.address !== '127.0.0.1'
        ) {
          out.push(iface.address);
        }
        return out;
      }, out);
    }, []);
  }
};

You can use such file from your web page with ease:

// create a NodeWorker instance
const nw = new NodeWorker('utils.js');
// setup an onmessage handler
nw.onmessage = (e) => {
  document.body.innerHTML = e.data.join('<br>');
};
// invoke some method
nw.postMessage({name: 'ipv4'});

Do you think ditching Electron and use this approach could be a valuable option?

I know Electron is cool and everything but it doesn't really work well on ARM, specially on Raspberry Pi Zero and/or 1, and on top of that, it needs Xorg or XWayland.

Pros Using WebKitGTK+ directly would bring in the following features:

• 100% ECMAScript 2015 support, including native JavaScript modules <script type=module>
• it's lightweight, it doesn't use as much as RAM Electron does, it doesn't need a parallel version of NodeJS to work
• the NodeWorker keeps everything easily contained in one folder while Electron cannot share same node_module folder with the OS
• boots on WebKitGTK+ and Wayland will take definitively less time
• you update the OS, not the OS plus the Electron plus rebuilt modules for either Electron or the OS

Cons

It's not Electron.

What do you think?

[maertz]

Awesome, sounds pretty neat. I'll give this definitely a try 👍

What about replacing "electron" with "electrician" to keep "B.E.N.J.A" ^^

For those which would like to stay with electron and are looking for a "node-worker" like thingy electron-remote might be an option.

[WebReflection]

I'll think about the name but I have a Raspberry Pi prototype ready to go.

cd ~/Downloads
curl -LOs http://os.archlinuxarm.org/os/ArchLinuxARM-rpi-2-latest.tar.gz
curlo -LOs benja.io/os/sh/setup-sd
chmod a+x setup-sd
./setup-sd /dev/sdX ArchLinuxARM-rpi-2-latest.tar.gz

Once finished, boot the RPi 2/3 and use alarm user to log in (same password).

You need a network too. Once logged in alarm:

curlo -LOs benja.io/os/sh/setup-system
curlo -LOs benja.io/os/sh/setup-user
bash setup-system

Once finished, just reboot :tada:

I cannot think at this point of anything lighter than that based on Web.

I'm now trying to figure out if same setup would work on a Raspberry Pi Zero :crossed_fingers:

[WebReflection]

FYI it works on Raspberry Pi Zero too but it's definitively clunky on a Full HD screen.

[maertz]

Awesome, I'm still struggling to flash my SD-Card on OSX since you posted the prototype - I hope I get there soon.

[WebReflection]

oh ... sorry, I'm on ArchLinux by default myself, no way my bash files will work on OSX :disappointed:

[WebReflection]

however, instructions to create the SD card are here https://archlinuxarm.org/platforms/armv7/broadcom/raspberry-pi-2

Things I've discovered meanwhile:

• youtube plays very badly on WebkitGTK, even if I've enabled all things I could enable
• audio doesn't play neither, while Electron and Qt WebEngine plays out of the box
• WPE looks abandoned or even slower than Benja itself at development progresses ... is there any RPi 2 updated ISO I can play with? NodeWolker should just work anywhere!

Thanks for joining me in this annoying journey :smile:

[maertz]

Sure mate 👍 Indeed annoying - especially if you get stuck by "simply" flashing a sd-card. Now I've just got that fancy idea to use a Linux VM to flash my card which feels quite cringy ^^

Anyways - does it make sense to try it with chromium instead of WebkitGTK, as I know electron uses chromium as well.

[WebReflection]

Qt WebEngine is chromium ... but it's full of configurations nightmare due missed Wayland support and poor Xorg support on RPi anyway ... kill me know!

If you want ArchLinux as VM give archibold.io a try: https://medium.freecodecamp.org/installing-arch-linux-from-scratch-b595095ddd48

[maertz]

Oh my bad didn't know that, I should definitely get more into linux stuff. Thanks for the archibold.io hint, works like a charm - awesome handy project 🎉

[WebReflection]

Update

I've managed to boot into QtWebEngine, which is based on Chrome 56, it scores 517 on html5test, and it's the lightest Chrome-like browser experience I could find around.

To make it happen, I've used QML (aka: QtQuick) to start the browser, and it just seems to works.

Pros

• it's the fastest browser I could bootstrap, faster than both WebKitGTK and Electron
• it has very good video playback performance. I could watch Youtube videos in fullscreen (Fulll HD) and when no ads were showing up, the quality was amazing and the audio was also working (via HDMI)
• it's not as memory greedy as Electron, even using gpu_mem=192 the default/demo Benja app left 630+ MB of free RAM while interacting with it
• it works with NodeWorker out of the box (but that's true for everything I've tried so far)
• it's part of regular ArchLinux distribution. No need shaka-laka alchemies rebuilding anything

Cons

• it doesn't ship with nodejs integrated (but NodeWorker fixes 90% of use cases)
• it doesn't render WebGL, even if force enabled. This might be fixed in November when Qt 5.10 goes out
• it needs xorg, although that's probably even better for LCD based on SP where I had to fallback to Xorg regardless, being still, and unfortunately, the most compatible target platform out there

BENJA (Bootable Electron ish Node JS Application) in QML

Grab the ArchLinux OS

cd ~/Downloads
curl -LOs http://os.archlinuxarm.org/os/ArchLinuxARM-rpi-2-latest.tar.gz

Please note the 64bit version has some issue with many modules ... it looks like nobody is testing anything in there, at least not in RPi3 so, for now, stick with version 2 of the OS (ARMv7)

Prepare the SD card (you need ArchLinux for this, I haven't tested anything else)

curlo -LOs benja.io/os/sh/setup-sd
chmod a+x setup-sd
./setup-sd /dev/sdX ArchLinuxARM-rpi-2-latest.tar.gz

Put the SD card in your PI, then do the following after logging in (alarm:alarm)

# you need network connection
curl -LO benja.io/os/qml/setup-system
curl -LO benja.io/os/qml/setup-user

bash setup-system

The root password is still root

Follow the initial instructions until sudo is installed, then you can do something else until it finishes (it takes a little while, depends on how fast is your SD card)

Once everything is done, type sudo reboot and that's it.

If you want to test different pages than the default one, just pass to the app/package.json script.start property a third argument like in:

{
  ...
  "scripts": {
    "start": "node index.js https://www.youtube.com/watch?v=nfWlot6h_JM"
  },
  ...
}

Please let me know if you'll have any issue with it. If not, I am planning to create a clean ISO with these steps pre-made and propose it in the website.

Regards

[WebReflection]

Memory staus on default / demo app

While playing Full HD Youtube video

[maertz]

Wow, awesome - pretty neat specs. I finally gave up flashing the SD card, I need to wait for the ISO :(

[WebReflection]

here the BENJA QML ISO for Raspberry Pi 2 and 3.

Let me know what you think, the concept is the same: plug in your laptop and change/edit the APP folder content but remove the node_modules folder before plugging it into the Raspberry Pi so it'll install modules for itself (unfortunately the socket.io wants a native module, maybe I should move that to the global dir instead of having it as local dependency)

ssh via alarm user instead of benja one, and the password is also alarm.

It's a proof of concept, but a very working one :smile:

[maertz]

Excitement! 🎉

[maertz]

Looks pretty cool, I've managed to get the GPIOs working - without the rebuilding hassle :)

Install yaourt https://wiki.archlinux.de/title/yaourt

install git yaourt -S --needed --noconfirm git

install python2.7 yaourt -S --needed --noconfirm python2

install PIGPIO

yaourt -S --needed --noconfirm python-pigpio-git

# copy libpigpio under /usr/lib
sudo cp /usr/local/lib/libpigpio* /usr/lib/

Install johnny five raspi-io

npm install --no-bin-links raspi-io
npm install --no-bin-links johnny-five

create test script

cd app/
touch test.js

try {
  const
    five = require('johnny-five'),
    Raspi = require('raspi-io'),
    board = new five.Board({
      io: new Raspi()
    })
  ;
  // somehow necessary to avoid the board to quit
  board.on('info', (e) => console.info(e.message));
  board.on('exit', (e) => console.log(e.message));
  // the entry point to use all GPIO in the RPI 2 or 3
  board.on('ready', () => {
    // get current led
    const led = new five.Led('P1-13');

    // switch the led on or off accordingly
    led.blink();
  });
} catch(e) {
  return [e.message];
}

Start test.js

sudo node test.js

Awesome, it works 👍 unfortunately I did not manage to start QML as root by simply adding sudo into alarm/.xinitrc to execute the test.js within the nodeWorker but anyways - awesome! Maybe it's possible to spawn a child process with root privileges within the nodeWorker.

[WebReflection]

install yaourt

bash <(curl -s archibold.io/install/yaourt)

make npm a super user

echo "${USER} ALL=NOPASSWD: $(which npm)"> sudoers
sudo bash -c 'cat sudoers >>/etc/sudoers'
rm sudoers

change .xinitrc

sed -i "s/npm start/sudo npm start/" ~/.xinitrc

And try again, it should work (please let me know if it doesn't)

[WebReflection]

also, about --no-bin-links stuff ... I don't really like it.

The reason is there is because the FAT32 partition hosting the APP directory is the best way I could think of delivering an easy to edit/test/deploy SD card: it works on every system.

However, wqhen it comes to native modules that inevitably needs node-gyp compilation, such universal FAT32 APP folder doesn't scale, which is why on BENJA the npm install --production --no-bin-links is used, so that modules in the devDependencies part of the package.json are not affected.

Ideally, I think it's fair to assume the following:

• in a non Electron environment, @webreflection/node-worker module is mandatory. It should be on the global npm directory, 'cause it needs compilation.
• the tiny-cdn helper is a very lightweight and super fast static content module server, but developers might want to use express, koa, whatever else instead, it should be a dependency that on BENJA should install without bin.
• so should be any other dependency: if it needs native code instead of just JS, it should be -g on Raspberry Pi and devDependency on package.json, in this way there should never be a package conflict if you test/copy/try your app directly from the SD card and you move it later on in the Pi.

Does any of this make sense?

Basically I'm willing to create an ISO that has johnny-five and raspi-io included so that nobody will have to go through this again ever.

[maertz]

lol ok, the archibold.io way is definitely easier 👍 regarding FAT32 I guess there is no other way to support a wide range of people to let them simply plug and play. If g33ks are getting annoyed of this they probably know how to fix/solve it by themselves.

Not sure how VM works within node-worker but if it spawns/runs inside the system enviroment maybe its worth a try to install johnny-five and raspi-io globally altogether with PIGPIO and try to make it work through that. If it works, it should work in electron as well without the struggle to recompile the stuff or am I wrong? (in case electron is still a thing here).

[maertz]

fyi on every touch app/reload it spawns a new qml :)

[maertz]

Some stats from unmodified ISO playing a youtube video in an iframe on fullscreen 1920x1080px - sound on both worked out of the box.

QML

480p Pretty smooth, plays well, looks good

720p Has some hickups, seems to heat up a lot

Electron

480p Pretty smooth but feels little sluggish

720p Has more hickups but I don't get that heat icon for some reason

[WebReflection]

fyi on every touch app/reload it spawns a new qml :)

OK, that's easy to fix via the APP folder, no need to create a new ISO.

The app demo is ... just a demo, but if you have hints to improve it, that's more than welcome.

Update

I'm uploading a new QML ISO after finally managing to login and sudo npm start right away without losing NODE_PATH informations.

Basically every user shares the same ~/.npm-global folder now, it's enforced in the /etc/environment and it just works.

This means that johnny-five, rapi-io, and @webreflection/node-worker are now pre-installed globally and the only (demo purpose) production module in the app is tiny-cdn.

Local modules in the app should be installed on the Pi like npm install --production --no-bin-links while modules that needs to be compiled should be installed as --save-dev in the package and just as npm install module -g in the pi.

Development through the APP folder is now easier, npm starts with sudo powers, QML is faster and lighter.

Can we call it a success? What else is missing in this ISO?

[maertz]

More memory, more power, super fast - sounds like an achievement :)

[WebReflection]

the latest ISO https://s3.amazonaws.com/benja.io/iso/benja-qml-rpi2.zip

[WebReflection]

FYI I have re-updated the ISO putting johnny-five example within the app ... when you see Ready to Rock on the screen it means the board is ready.

You can either flush it again or grab latest files from the repo:

The new worker: https://github.com/WebReflection/benja/blob/gh-pages/os/qml/workers/utils.js

The new index: https://github.com/WebReflection/benja/blob/gh-pages/os/qml/index.html

Also the new index.js: https://github.com/WebReflection/benja/blob/gh-pages/os/qml/index.js

There is a module to install for the demo (or again, just grab the latest ISO)