headless version

[megablue]

It would be great that if node-webkit are made into headless version...

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[Mithgol]

You mean, like, the same Node.js engine and API, but without any browser-alike HTML+CSS interfaces?

But that's what original Node.js is. Maybe you should just get it instead of node-webkit then.

[tommoor]

Sounds like you're looking for www.phantomjs.org

[dhendo]

Or https://github.com/deanmao/node-chimera/

[ronward]

I would very much like to see a headless option for node-webkit. And yes I know that is like phantomjs, but phantomjs does not have the nodejs integration that node-webkit has. Also it can them mean just the one code base and not having to testing to see which environment you are running under etc.
A must have feature! IMHO

[szwacz]

I didn't get the point of this feature. What exactly you are expecting out of headlessness? Why today node-webkit with hidden main window is not the solution?

[ronward]

The reason that node-webkit with just the main window hidden is not a solution is because it still expects a display (XWindow/XOrg). e.g. you cannot run it on a server because there is no display. (this is on Linux) When I try this I get the message: Gtk-WARNING **:cannot open display [25522:0100/000000:ERROR:zygote_linux.cc(498)] write: Broken pipe

[katanacrimson]

@ronward ...then just run node?

[ronward]

But node does not have the browser context! What I want is more like PhantomJS (yes I know, then just use PhantomJS), but it would be very nice to have the combination of node and the browser context (e.g. node-webkit) in one. (and there may be other solution e.g. https://github.com/deanmao/node-chimera/ as mentioned above, but this would mean rewriting code or having two code bases)

[rogerwang]

This is on my plan. Before that you can use xvfb, xvnc or other X server which don't need a physical display on your server.

[ronward]

Thanks, that a very good idea that I had completely forgotten about. (used xvfb to run OpenOffice ages ago when there headless option still had some bugs) By the way I just love the node-webkit idea/system/setup! Thanks

[kaizhu256]

+1 valid use cases for a headless node-webkit are:

1. provide automated browser testing and code coverage on a build server
2. running a web-scraper on a server

[edwardchanjw]

headless should help develop background apps? or We can set show to false in the manifest of your application with similar purpose?

[tommoor]

To me this dilutes the projects aims. Phantomjs does absolutely fine for 99% of purposes.

[Mithgol]

I was the first here to point to another project that looks like a headless node-webkit, but today I myself encountered a perfectly valid case for using a headless node-webkit — a case where nothing else would suffice.

That's Travis CI testing of a Node.js addon-containing module. There's no other way to test if a module runs on node-webkit; the module has to be built by nw-gyp and run on node-webkit to see what happens.

And, almost like @ronward said above, I saw precisely the following:

node-pre-gyp ERR! stack Error: Command failed:
node-pre-gyp ERR! stack (nw:2106): Gtk-WARNING **: cannot open display:
node-pre-gyp ERR! stack [2109:0100/000000:ERROR:zygote_linux.cc(478)] write: Broken pipe

You also may look into the my Travis CI log (lines 1271—1273) for an example.

[Mithgol]

@rogerwang

Before that you can use xvfb, xvnc or other X server which don't need a physical display on your server.

Sounds like a good plan.

Travis CI even provides a script for starting xvfb.

Now how do I run node-webkit on xvfb?

(Unless it already does that automagically based on $DISPLAY, which I am going to check soon.)

[Mithgol]

Got Package appears valid on this log, line 1272. It seems that xvfb is enough for a headless node-webkit on a Travis CI server.

[Mithgol]

Running 32bit node-webkit on 64bit Travis CI Linux is also possible (though a bit tricky), see #1566 for details.

[FWeinb]

@Mithgol I get a:

Xlib:  extension "RANDR" missing on display ":99".
[1183:0215/113833:ERROR:breakpad_linux.cc(1225)] crash dump file written to /tmp/chromium-renderer-minidump-5d9a40a22d7d15b8.dmp

when running v0.9.1 - Feb 11, 2014(64bit) on a digitalocean 5$ instance. I started xvfb like this:

/usr/bin/Xvfb +extension RANDR :99 -ac -screen 0 1024x768x8 &

Any idea?

[Mithgol]

No idea, sorry. My experience does not tell me anything about that. Two days ago I've run xvfb for the first time in my life. I suppose there's some difference between digitalocean and Travis, because in Travis docs they write something very similar to your attempt (except that +extension RANDR thing and bit depth).

[FWeinb]

Thanks for your quick answer. I could get it to work. There where some font's missing

sudo apt-get install -y xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic
sudo apt-get install -y x-ttcidfont-conf cabextract ttf-mscorefonts-installer # accept the EULA
sudo dpkg-reconfigure --default-priority x-ttcidfont-conf

The +extension RANDR isn't needed.

[cretz]

For those looking for another usecase here, a headless WebRTC peer would be very nice. From what I can tell, PhantomJS does not support this Chromum feature and https://github.com/js-platform/node-webrtc seems not to be up to par yet for cross-OS and easy installation for users. I very much look forward to headless node-webkit.

[CooCooCaCha]

Another future use-case would be for PDF rendering. PhantomJS is nice but it is dependent on QTWebkit for updates. I like the idea of doing PDF rendering from a library that runs on chromium and node than one that runs on QTWebkit.

[Mithgol]

A brief follow-up on the RANDR problem. Todd Wolfson says that it's fine to ignore the Xlib error about RANDR. I guess we all have to expect (from now on) that stderr of node-webkit can be non-empty even if everything else is fine (including the application's exit code).

[Mithgol]

Also, I tried +extension RANDR on Travis CI's version of Xvfb today.

It does not seem to prevent the same node-webkit's RANDR-related Xlib error.

Useless.

[Starcounter-Jack]

Reasons that Node-Webkit would be the better headless browser:

1. PhantomJs is based on Webkit whereas Node-webkit is based on Blink. Will Apple maintain Webkit on non Apple platforms? Opera, Qt and others are moving away and Safari is discontinued on non Apple platforms.
2. V8 performance is much better than JavaScriptCore in PhantomJs
3. One of the uses of headless browsers is crawlable Ajax. PhantomJs does not support web components or Polymer making PhantomJs a non-option for this use case.

[jcrubino]

With WebRTC entering the mainstream a headless chromium api would help create services based on WebRTC. Just saying there is a need not what the best solution is.

[Rameshv]

+1

[tbranyen]

"To me this dilutes the projects aims. Phantomjs does absolutely fine for 99% of purposes."

This is not a valid counter argument. In what way does PhantomJS share a V8 context with Node as the project aim of Node-webkit does?

[chriddyp]

I just got node webkit running off a fresh amazon ec2 server running ubuntu 12.04.

Here is my complete terminal history to get this running. Thanks for all the help on this thread!

# Freshen up ubuntu
sudo apt-get update
sudo apt-get dist-upgrade

# Grab node - add it to the apt listings then install
sudo add-apt-repository ppa:chris-lea/node.js
sudo apt-get update
sudo apt-get install nodejs

# get xvfb
sudo apt-get install xvfb

# get n unpack node-webkit
wget http://dl.node-webkit.org/v0.9.2/node-webkit-v0.9.2-linux-x64.tar.gz
tar -xvzf node-webkit-v0.9.2-linux-x64.tar.gz 

# some initial depencies
sudo apt-get update && sudo apt-get install -y xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic x11-apps imagemagick

# a lot of libraries are missing (libXtst, libgtk2.0, libnss3, libgconf2-4, libasound2)
# I had success in just installing chrome instead of trying to install the libraries
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
sudo dpkg -i google-chrome-stable_current_amd64.deb
sudo apt-get -f install
sudo dpkg -i google-chrome-stable_current_amd64.deb

## install some fonts
# to install the mscorefonts, allow apt to grab files from multiverse
# Do this by uncommenting two lines in /etc/apt/sources.list:
# deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ precise multiverse
# deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ precise-updates multiverse
sudo -e /etc/apt/sources.list
sudo apt-get update
sudo apt-get install ttf-mscorefonts-installer
sudo apt-get install -y x-ttcidfont-conf cabextract
sudo dpkg-reconfigure --default-priority x-ttcidfont-conf

# run a simple example with xvfb - running `hellonodewebkit` here
sudo xvfb-run --server-args='-screen 0, 1024x768x24' node-webkit-v0.9.2-linux-x64/nw hellonodewebkit

# restart by killing xvfb and nw
pkill [X,x]vfb
pkill nw

[katanacrimson]

...sudo xvfb-run? Is root really required there? Also, dist-upgrade in that right off the bat, not the best idea.

[guybrush]

node is not chromium at all. no webgl, no webrtc, no indexeddb, no html5 in general. and phantom has no node/module-system.

so a headless node-webkit would be cool for a lot of usecases.

another idea: being able to use webworkers (with transferable objects) would be cool on the server :D

[jure]

Agreed, headless node-webkit would be cool. A project I'm currently working on would benefit from the ability to run a headless WebRTC peer and use Chromium's storage APIs, so :+1: from me.

[warpech]

+1

[yuriyp]

+1

[opensourcegeek]

+1, in my case I want to run node.js as a stand alone binary which pulls in all the required binary modules as well. Something like nexe which bundles in binary modules too perhaps headless node-webkit might be an overkill?

[buschtoens]

:+1:

[pirog]

+1

[michahell]

late, but +1 for building GUI apps using CI systems in turn using a headless node-webkit. awesomesauce defined.

[johntitus]

+1 as a step towards a replacement of phantomjs.

[michalliu]

+1 for this feature. I want to get rid of XServer to run node-webkit

[lynchpin4]

this would be a beautiful thing, perhaps looking into what phantomjs did in the early days? (i think it was a simpler patch that didn't completely remove the xserver dependency, but didn't require a screen)

:+1: for the team. phantomjs has never been a preferential tool for me due to the loss of flexibility having a strictly headless browser. ideally when things don't work i'd like to see the state of the page / debug with devtools. would make development of such automated testing scripts much easier from a workflow point of view.

[DinisCruz]

+1 for the info on this thread (which help me when trying to run node-webkit without seeing the UI and in linux/travis scenarios)

Here is how I did it: https://github.com/rogerwang/node-webkit/issues/1966#issuecomment-65922211 and https://github.com/rogerwang/node-webkit/issues/1966#issuecomment-65923266

[simonbreak]

+1 so I can use Web Audio API from node

[sheerun]

+1 phantomjs seem to have quite outdated WebKit. node-webkit is real-life Chromium.

[Sesshomurai]

A fully browser compliant, headless mode node-webkit that can run web pages (including all the javascript in them) without xvfb or any GUI would be a great and wonderful thing. Nothing like this exists (that runs on node) and who wants to juggle 5 different tools to get close to doing this anyway. Please make this happen. The new possibilities will be exciting. Maybe there is a special "headless" renderer needed, that does not require any GUI of any kind.

[DinisCruz]

@Sesshomurai and others, so what is the definition of 'headless'?

Is it that it can run on server? or that it can run without been seen?

For both those scenarios I was able to make it work with https://github.com/o2platform/nwr and https://travis-ci.org/o2platform/nwr (travis is running the tests using xvfb)

[Sesshomurai]

@DinisCruz. To me, true headless is A) run on a server B) Does not require X11 or any graphical libraries (which xvfb does) C) Does not expose any UI (either virtual or real). D) Is optimized. Yes, there may be workarounds, but a native headless solution is better (and node-webkit is 9/10ths of the way there). xvfb uses memory for screen buffers.

[EthraZa]

I have an app that runs on Windows and Linux. But, everyone that think in run it on Linux, think about it as a service, starting at boot. The UI would be only to the one time configuration stuff. So, I too think that a headless without xvfb nw, that just works, would be very cool indeed.

[loveencounterflow]

@sheerun one thing that didn't work in phantomjs years ago and still didn't when i checked a few weeks ago is Unicode code points beyond 0xffff. This is unacceptable for me and yes it feels outdated.

And just as @gstack and other have said there's a huge difference between a strictly headless browser and a browser that has the ability to run headless. To say that 'we don't need a headless nw we already have headless browsers' makes about as much sense as saying 'we don't need nodejs because JS already runs in the browser'. You either subscribe to both statements or to none if you want to make any sense.

To make another point, my use case for a nw-without-gui is rendering page contents to image / pdf files; for this i don't need user interaction which is great because i can then have a server or a Vagrant terminal-only VM do the work. But i do need a display of the pages as long as i'm still fiddling with the layout. I did it before with phantom.js but that's a pain, not only for the lacking Unicode support but also because you have to either open a 'real' browser to get a look onto your design, or have phantom output some pages and then look at the images. Both ways are far from optimal: You can't interact with a static image the way you can interact with a page in a web browser with dev tools, and using another browser means your display will never be accurate. Never. Other browser, other version, there will be differences.

[Sesshomurai]

@loveencounterflow Agree 100%. My use case is similar. I want to run the same code that runs in the browser in a node.js app that executes as a real browser sandbox. My javascript code is a lot of socket.io stuff and generally doesn't touch the DOM, but the client code for socket.io is entirely different within the browser vs. the node.js client package. Different code. So now I have to make a separate codebase and client to do the same thing to spawn off in a bunch of node apps to emulate browser connections. And that doesn't really test my browser-side code, does it? Because its not the exact same code nor a true browser sandbox. I think projects like this (node webkit) are "getting close" to realizing the need for a truly headless browser in Node. It would require interfacing with some frame buffer library I'm sure. But hopefully we can get there.