How to: RaspberryPi Headless Mode (no GUI/Window Manager)

[TibGva]

Hello,

I am trying to use this node, as it seems a great tool. I start with a very very basic flow:

[{"id":"656335c1.fd2edc","type":"inject","z":"82b705f6.4ea288","name":"","topic":"","payload":"http://www.google.fr","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":250,"y":360,"wires":[["cd60f341.0223e","fa8e65e6.6f3f78"]]},{"id":"d7ea3f30.0787","type":"debug","z":"82b705f6.4ea288","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"nbrowser","x":800,"y":360,"wires":[]},{"id":"cd60f341.0223e","type":"debug","z":"82b705f6.4ea288","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":530,"y":280,"wires":[]},{"id":"fa8e65e6.6f3f78","type":"nbrowser","z":"82b705f6.4ea288","name":"","methods":[{"name":"gotoURL","func":"goto","params":[{"type":"msg","value":"payload","typeDefault":"str"}]}],"prop":"nbrowser","propout":"payload","object":"msg","objectout":"msg","close":false,"show":false,"ssl":false,"outputs":1,"x":520,"y":360,"wires":[["d7ea3f30.0787","83da0c88.0c242"]]},{"id":"83da0c88.0c242","type":"debug","z":"82b705f6.4ea288","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":820,"y":440,"wires":[]}]

but I do not receive anything in outputs. I am just trying to receive the page or do a search... Any help would be welcome ;-)

[balsimpson]

My flows are also not working.

[Steveorevo]

It looks like the method is hanging; the browser is not finished or hangs loading the given URL. Does the browser even appear with the option “Show browser window instance?” checked?

Also, are their any error messages in the output console when starting Node-RED from the command line?

Lastly, what Window manager are you using (ie macOS, Windows 10, Ubuntu Desktop, etc. )?

[TibGva]

No, the browser does not appear, but my node-red is hosted on AWS ubuntu, I am accessing it from another machine using chrome browser on macOS.

[Steveorevo]

The browser window would open on the desktop of the hosting machine; if you even have a desktop. It sounds like you do not have a window manager; a hard requirement for using browser.

See last comment in #5

You need a “window manager” or could install xvfb.

[cassiopeiae]

Same for me ... does not work at all. Hangs when using a simple gotoURL and even does not ouput anything on console. Good idea but useless...

[Steveorevo]

Are you also not meeting the requirements?

[rybycy]

Guys, I actually managed to get it working on my RPi.

First step: use xvfb - apt-get install xvfb I'v created a simple node-red node placed right before nbrowser - feel free to reuse https://gist.github.com/rybycy/ebba8b0e1bd0b874f7282f4dc5e68fa1 (quite dirty, works though). Please follow official path of registering new nodes to finish that.

Second step: how to get rid of Navigation timed out after 30000 ms https://github.com/segmentio/nightmare/issues/1415

Solution: Update Electron used by Nightmare I've done that from <your-node-red-user>/.node-red/node_modules level: npm install electron@1.7.12.

For me it solves the issue completely

P.S. @Steveorevo thank you sir for your work!

[johnmckerrell]

@rybycy I'm struggling to get this working, can you provide some more insight? I'm afraid I don't have much experience of Node-RED but I think I've done what you said:

• Installed xvfb: apt-get install xvfb

• Created my own Node-RED module for xvfb: https://gist.github.com/johnmckerrell/54877cd0e15a7f4364736458f2752f96

• Installed v1.7.12 of electron:

  cd ~/.node-red/node_modules/
  npm install electron@1.7.12

I've then got a flow as follows:

• string inject node of basic URL
• Start XVFB node
• nbrowser node
  • gotoURL - msg.payload
  • getHTML - [blank for all]
• debug node outputting the whole msg object.

Once deployed I click to inject the URL, status of the nbrowser goes through "running: gotoURL" to "running: getHTML" but nothing shows up from the debug mode.

Any suggestions?

[johnmckerrell]

In case anyone ends up here with similar problems to me, I went through a debugging process of trying to get nightmare.js examples to run on the command line, failing, finding out about DEBUG=nightmare:*,electron:* node example.js (i.e. the DEBUG arg gives lots of useful info), found out I was missing a dependency, running the following installed the dependencies:

sudo apt-get install build-essential clang libdbus-1-dev libgtk-3-dev libnotify-dev libgnome-keyring-dev libasound2-dev libcap-dev libcups2-dev libxtst-dev libxss1 libnss3-dev gcc-multilib g++-multilib curl gperf bison python-dbusmock openjdk-8-jre

Restarted node-red to make sure and then tried my flow and it worked without a hitch.

[balsimpson]

I tried all the steps. Debug node is still silent. I was unable to install gcc-multilib though.

[johnmckerrell]

@balsimpson I just had to setup my install again with some difficulties, it turned out that I hadn't noticed I was running quite an old raspian, I think I had just used what came with the Pi and it wasn't very up to date. On my new box I used the latest raspbian, and the apt-get install line above didn't work, looking at my history I now have:

sudo apt-get install build-essential clang libdbus-1-dev libgtk-3-dev libnotify-dev libgnome-keyring-dev libasound2-dev libcap-dev libcups2-dev libxtst-dev libxss1 libnss3-dev  curl gperf bison python-dbusmock openjdk-8-jre libgconf-2-4

I had previous used a symlink for my xvfb node module but that just wasn't picked up, so I ended up moving it into ~/.node-red/node_modules and everything worked.

[Dimey]

@johnmckerrell Can you tell me, which version of raspbian you are using? I think I have a very basic problem with a symlink to xvfb. But moving it to ~/.node-red/node_modules doesn't solve the problem, still can't find the module 'xvfb'.

[johnmckerrell]

@Dimey this is what I have:

PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"

and

pi@mqtt:~/.node-red $ ls -l node_modules/ | grep xvfb
lrwxrwxrwx  1 pi pi    12 Apr 10 20:18 node-red-contrib-mcknut-xvfb -> ../xvfb-node
drwxr-xr-x  2 pi pi  4096 Mar 31 20:28 xvfb

(the lower one obviously being the main node package)

I've been having real difficulties with my setup so as you can see I've tried moving the module to a symlink but within the .node-red folder. I tried running npm update and upgrading everything recently but ended up with problems again. As such I think I'm currently running the node packages that I installed months ago on my old Pi so if you're doing it from fresh you might be getting different, incompatible, packages. I'll attach my package.json files for comparison.

package.json.txt package-lock.json.txt

I think most of my difficulties now are around low system resources, I tried installing a DVR system on the Pi as well, and also syncthing. Poor little Pi isn't really up to being a full server, I'll probably switch to a 4GB Pi 4 soon.

[Dimey]

@johnmckerrell Thank you for your reply. Much appreciated. package.json and package-lock.json are pretty identical to mine.

A strange circumstance is, that apt-get install xvfb doesn't create a xvfb folder in _/home/pi/.node-red/nodemodules . So I tried npm install xvfb@0.3.0, it created a folder named xvfb. But with node-red-start there is still the message

11 Apr 16:05:09 - [warn] [node-red-contrib-mcknut-xvfb/xvfb] Error: Cannot find module 'xvfb'

So my problem begins a bit earlier at the stage of registering a custom node. Can't figure out why my system is not able to find 'xvfb' or even installing it at the correct place.

I'm using a fresh pi 4 system with the newest raspbian.

[johnmckerrell]

@dimey apt-get install xvfb is installing the linux package, then you've got the node package and the node-red package to worry about as well! Do you have anything in .node-red/node_modules/xvfb ? That's the node package which you need, and then the node-red package is the one that I've put in my gist.

[Dimey]

@johnmckerrell OK, the node registration process is fixed by putting your contrib-folder directly to _~/.node-red/nodemodules/. I can now select Start Xvfb and End Xvfb Nodes in the palette. But I guess, something is still wrong with my xvfb installation because electron is not starting and no output at the debug node.

This is the folder structure:

.node-red
|- node-red-contrib_mcknut-xvfb
|    |- package.json
|    |- xvfb.js
|    L xvfb.html
|
|- ...
L xvfb
     |- index.js
     |- LICENSE
     |- package.json
     L README.md

Does it look like your structure?

[kilbamoo]

Same error here: [warn] [node-red-contrib-mcknut-xvfb/xvfb] Error: Cannot find module 'xvfb' xvfb folder exists in node_modules folder and has packages, not empty. Did you solve @Dimey ?

edit Ok, got it running. xvfb needs to be installed inside mcknut-xfvb folder...

[Dimey]

@kilbamoo Thanks for your solution. Worked out for me as well.

[azrin1972]

@johnmckerrell I've used your steps but not on respbarrybut on Ubuntu server instead.

I'm not getting anything from my debug node.

Here is my steps

An injector with URL START XVFB NBROWSER Debug

I've install xvfb npm and electron but i got error electron module not found.

when do we use the END XVFB node by the way?

[johnmckerrell]

Did you install xvfb from apt? Just you don't specifically mention that just the npm module. Actually you mention "electron module not found" do you have the electron npm module installed?

You don't particularly need the END XVFB node if you're going to keep the electron node around. If you want to clean everything up you would check the "Close instance after methods?" in the browser node and also use an END XVFB node to clean that up.

[azrin1972]

Did you install xvfb from apt? Just you don't specifically mention that just the npm module. Actually you mention "electron module not found" do you have the electron npm module installed?

You don't particularly need the END XVFB node if you're going to keep the electron node around. If you want to clean everything up you would check the "Close instance after methods?" in the browser node and also use an END XVFB node to clean that up.

Yes I did install xvfb from apt and later manually install electron but the latest release is v9.x

[johnmckerrell]

Just leaving a note here that I have actually now published the XVFB Node-RED module as you can see here: https://flows.nodered.org/node/node-red-contrib-xvfb

This should cause the XVFB NPM dependency to be installed but you will still need to ensure that you have installed XVFB itself using Apt, Yum or whatever makes sense for your system.