Error: "what(): Failed to open X display" in Ubuntu VM

[Mariusio]

I installed "mapbox-gl-native" for node in my Ubuntu virtual machine, however when I try to run the sample code (https://github.com/mapbox/mapbox-gl-native/tree/master/platform/node#rendering-a-map-tile), I get the error

terminate called after throwing an instance of 'std::runtime_error'
  what():  Failed to open X display.
Aborted (core dumped)

My system setup:

node: 5.4.1
mapbox-gl-native: 3.1.2 (build from source without errors)
sharp 0.14.1
Ubuntu: 15.10 running in a Virtualbox (5.0.16) on an OSX host system.

Can you give me any hints how to solve this? Is it possible to run mapbox-gl-native in a VM and export map images using node?

[mikemorris]

Yes, it's possible (I ssh into a Linux VM setup for local Node.js mapbox-gl-native dev), but it can be a bit painful dealing with configuration and Linux graphics driver issues. You can check in scripts/travis_setup.sh for some hints, but off the top of my head here's the most important bits:

• Start up a real X server (by installing the X.org drivers on your host Mac and sshing into the VM with -X or -Y) OR spin up xvfb on the Linux VM for a virtual framebuffer

On the Linux VM:

• Install a relatively recent version of Mesa on the Linux VM (we're using 10.4.3 for CI testing with these build flags)
• export DISPLAY=:0 for real X server (I think) or export DISPLAY=:99.0 for xvfb
• export LD_LIBRARY_PATH="{PATH_TO_MESA_LIB_DIRECTORY}"

[mikemorris]

Oh, and as a final note, this configuration requires a hardware GPU on the host machine. I started exploring software rendering with OSMesa/OpenSWR in https://github.com/mapbox/mapbox-gl-native/pull/4638 but that's not complete yet.

[Mariusio]

Thanks a lot for the replies. I tried your recommendations and now I don't get any errors anymore. However, when running this code https://github.com/mapbox/mapbox-gl-native/tree/master/platform/node#rendering-a-map-tile I also don't get any output at all. The script doesn't seem to finish. Could you point me to some sample code that would just export a map as image so that I can verify whether my setup works now? Thanks for the help, highly appreciated.

[mikemorris]

@Maarius Can you figure out where the script hangs? Does map.render ever return, or is the issue in image.toFile? It may also help to listen to log events, to see if there's an issue loading resources or something.

[Mariusio]

It looks like map.render hangs. I experience the same behavior even when all the code inside of map.render is commented out. However, if I comment out map.load(require('./test/fixtures/style.json')); I get immediately the error TypeError: Style is not loaded. So it seems the script is executing something. I added the listening to log events but there are no messages at all.

This is the result of cloning mapbox-gl-native and running node /mapbox-gl-native/platform/node/test/render.test.js

* skipped geojson clustered
* skipped image default
* skipped image raster-brightness
* skipped image raster-contrast
* skipped image raster-hue-rotate
* skipped image raster-opacity
* skipped image raster-saturation
* skipped image raster-visibility
* skipped line-triangulation default
* skipped line-triangulation round
* skipped video default
* passed background-color default
* passed background-color function
* passed background-color literal
* passed background-color overlay
* passed background-opacity color
* passed background-opacity image
* passed background-pattern @2x
* passed background-pattern literal
* passed background-pattern pitch
* passed background-pattern rotated
* passed background-pattern uneven-pattern
* passed background-pattern zoomed
* passed background-visibility none
* passed background-visibility visible
* passed circle-blur default
* passed circle-blur function
* passed circle-blur literal
* passed circle-color default
* ignore circle-color feature-function
* passed circle-color function
* passed circle-color literal
WARNING (ParseStyle): zoom level in stop must be a number
* ignore circle-color zoom-and-feature-function
* passed circle-opacity default
* ignore circle-opacity function
* ignore circle-opacity literal
* passed circle-radius default
* passed circle-radius function
* passed circle-radius literal
WARNING (ParseStyle): zoom level in stop must be a number
* ignore circle-radius zoom-and-feature-function
* passed circle-translate default
* passed circle-translate function
* passed circle-translate literal
* passed circle-translate-anchor map
* passed circle-translate-anchor viewport
* passed classes additive
* passed classes override
* ignore debug collision
* ignore debug collision-overscaled
* ignore debug tile
* ignore debug tile-overscaled
* passed empty empty
* ignore extent 1024-circle
* ignore extent 1024-fill
* ignore extent 1024-line
* ignore extent 1024-symbol
* passed fill-antialias false
* ignore fill-color default
* ignore fill-color function
* ignore fill-color literal
* ignore fill-color multiply
* ignore fill-color opacity
* ignore fill-opacity default
* ignore fill-opacity function
* ignore fill-opacity literal
* ignore fill-outline-color default
* ignore fill-outline-color function
* ignore fill-outline-color literal
* ignore fill-outline-color multiply
* ignore fill-outline-color opacity
* passed fill-pattern @2x
* ignore fill-pattern literal
* ignore fill-pattern opacity
* passed fill-pattern uneven-pattern
* passed fill-pattern wrapping-with-interpolation
* passed fill-pattern zoomed
* ignore fill-translate default
* ignore fill-translate function
* ignore fill-translate literal
* ignore fill-translate-anchor map
* ignore fill-translate-anchor viewport
* passed fill-visibility none
* passed fill-visibility visible
Segmentation fault (core dumped)

Could the Segmentation fault (core dumped) be a pointer to the hanging script?

[Mariusio]

In order to verify that this is not an issue with my local VM I just created a new server and ran the same steps. The behavior is exactly the same as on my local machine.

[mikemorris]

@Maarius Hrm, well if you're seeing a segfault then something is going wrong. Could you run the render test under gdb to grab a backtrace?

gdb node --args /mapbox-gl-native/platform/node/test/render.test.js
r
thread apply all bt

[Mariusio]

@mikemorris I tried to grab a backtrace but somehow didn't get it to work. Will try again tomorrow. I have the feeling am getting closer thought. With the following code I get already an image file, however it is completely blank and I get this error in the logs.

{ class: 'ParseStyle',
  severity: 'ERROR',
  text: 'Error parsing style JSON at 0: Invalid value.' }

var mbgl = require('mapbox-gl-native');

var mbgl = require('mapbox-gl-native');
mbgl.on('message', function(msg) {
    console.log('%s (%s): %s', msg.severity, msg.class, msg.text);
});

var request = require('request');
var url = require('url');
var sharp = require('sharp');
var util = require('util');
var tilelive = require('tilelive');
var tilejson = require('tilejson');
var mapbox = require('mapbox');
require('tilejson').registerProtocols(tilelive);
var style = 'mapbox://styles/mapbox/streets-v8';

var fs = require('fs');
var path = require('path');

require('mapbox-gl-styles');

var options = {
    request: function(req, callback) {
        var opts = {
            url: req.url,
            encoding: null,
            gzip: true
        };

        if (url.parse(req.url).protocol === 'mapbox:') {
            opts.qs = { access_token: process.env.MAPBOX_ACCESS_TOKEN};
        }

        request(opts, function (err, res, body) {
            if (err) {
                callback(err);
            } else if (res.statusCode == 200) {
                var response = {};

                if (res.headers.modified) { response.modified = new Date(res.headers.modified); }
                if (res.headers.expires) { response.expires = new Date(res.headers.expires); }
                if (res.headers.etag) { response.etag = res.headers.etag; }

                response.data = body;

                callback(null, response);
            } else {
                console.log(body);
                callback(new Error(JSON.parse(body).message));
            }
        });
    },
    ratio: 1
};

var map = new mbgl.Map(options);

map.load(style);

map.render({
  width: 2048,
  height: 2048
}, function(err, pixels) {
  map.release();

  var image = sharp(pixels, {
      raw: {
          width: 2048,
          height: 2048,
          channels: 4
      }
  });

  // Convert raw image buffer to PNG
  image.toFile('image.png', function(err) {
      if (err) throw err;
  });
});

Could you help me once more and tell me how to adjust this code in order to make it work? Thank you so much for your help.

[mikemorris]

Just a quick glance, but it looks like you might be hitting https://github.com/mapbox/mapbox-gl-native/issues/3862#issuecomment-207631064

[Mariusio]

That looks like a possibility. I adjusted the code to this:

var mbgl = require('mapbox-gl-native');
mbgl.on('message', function(msg) {
    console.log('%s (%s): %s', msg.severity, msg.class, msg.text);
    console.log(msg);
});
var request = require('request');
var url = require('url');
var sharp = require('sharp');
var util = require('util');
var tilelive = require('tilelive');
var tilejson = require('tilejson');
var mapbox = require('mapbox');
var mbutil = require("mapbox-gl/js/util/mapbox");
var fs = require('fs');
var path = require('path');

var accessToken = 'pk.xxx'

var style = url.parse(mbutil.normalizeSourceURL('mapbox://mapbox.mapbox-streets-v6', accessToken));

var options = {
    request: function(req, callback) {

      var opts = {
          url: req.href,
          encoding: null,
          gzip: true
      };

      request(opts, function (err, res, body) {
          if (err) {
              callback(err);
          } else if (res.statusCode == 200) {
              var response = {};

              if (res.headers.modified) { response.modified = new Date(res.headers.modified); }
              if (res.headers.expires) { response.expires = new Date(res.headers.expires); }
              if (res.headers.etag) { response.etag = res.headers.etag; }

              response.data = body;

              callback(null, response);
          } else {
              console.log(body);
              callback(new Error(JSON.parse(body).message));
          }
      });
    },
    ratio: 1
};

var map = new mbgl.Map(options);

map.load(style);

map.render({
  width: 2048,
  height: 2048
}, function(err, pixels) {
  if (err) throw err;

  map.release();

  var image = sharp(pixels, {
      raw: {
          width: 2048,
          height: 2048,
          channels: 4
      }
  });

  // Convert raw image buffer to PNG
  image.toFile('image.png', function(err) {
      if (err) throw err;
  });
});

I don't receive the error anymore, but the generated image is still empty. Do you have more ideas?

[Mariusio]

The url that mbutil generates seems to be correct. If I open it in a browser I see a large JSON file.

[Mariusio]

Okay, some progress. When I use var style = require('./test/fixtures/style.json'); with

{
   "attribution":"<a href=\"https://www.mapbox.com/about/maps/\" target=\"_blank\">&copy; Mapbox</a> <a href=\"http://www.openstreetmap.org/about/\" target=\"_blank\">&copy; OpenStreetMap</a> <a class=\"mapbox-improve-map\" href=\"https://www.mapbox.com/map-feedback/\" target=\"_blank\">Improve this map</a>",
   "bounds":[
      -180,
      -85.0511,
      180,
      85.0511
   ],
   "center":[
      0,
      0,
      0
   ],
   "created":1428206400000,
   "description":"",
   "filesize":0,
   "fillzoom":8,
   "format":"pbf",
   "id":"mapbox.mapbox-streets-v6",
   "mapbox_logo":true,
   "maskLevel":8,
   "maxzoom":15,
   "minzoom":0,
   "name":"Mapbox Streets V6",
   "private":false,
   "scheme":"xyz",
   "tilejson":"2.0.0",
   "sources": {
     "mapbox": {
       "type": "vector",
       "maxzoom": 15,
       "tiles": [
          "https://a.tiles.mapbox.com/v4/mapbox.mapbox-streets-v6/{z}/{x}/{y}.vector.pbf?access_token=pk.xxx"
       ]
     }
   },
   "layers": [
     {
       "id": "background",
       "type": "background",
       "paint": {
         "background-color": "white"
       }
     },
     {
       "id": "water",
       "type": "fill",
       "source": "mapbox",
       "source-layer": "water",
       "paint": {
         "fill-color": "blue"
       }
     }
   ],
   "version": 8,
   "webpage":"https://a.tiles.mapbox.com/v4/mapbox.mapbox-streets-v6/page.html?access_token=pk.eyJ1IjoibW9laCIsImEiOiJXU1JqOW5NIn0.boeDGBoDtp1I0HOW-UaE0g"
}

as style.json, I at least get an image. The last thing that is missing is now being able to use the spec from https://www.mapbox.com/mapbox-gl-style-spec/. If I include an JSON with such format I get the error Error: Invalid protocol: mapbox:. Any hints how to solve this? I think then I would be done.

[Mariusio]

I was able to convert the URLs based on the comment you pointed me to in your last post.

[Mariusio]

@mikemorris I have one remaining issue and would be extremely grateful if you helped me. I would like to use this style to generate a map: https://github.com/mapbox/mapbox-gl-styles/blob/master/styles/basic-v9.json.

Using mbutil, I convert the mapbox:// links to https:// links, so the result is:

sourceUrl (input):
mapbox://mapbox.mapbox-streets-v7
sourceUrl (output):
https://api.mapbox.com/v4/mapbox.mapbox-streets-v7.json?access_token=pk.xxx&secure

spriteUrl (input):
mapbox://sprites/mapbox/basic-v9
spriteUrl (output):
https://api.mapbox.com/styles/v1/mapbox/basic-v9/sprite?access_token=pk.xxx

glyphsUrl (input):
mapbox://fonts/mapbox/{fontstack}/{range}.pbf
glyphsUrl (output):
https://api.mapbox.com/fonts/v1/mapbox/{fontstack}/{range}.pbf?access_token=pk.xxx

The links seem to be correct, as I can open them in a browser and get the JSON or image. However, If I replace now the mapbox:// urls in the style.json with the generated links and run my script, I get the error:

ERROR (Style): Failed to load sprite: unsupported image type
{ class: 'Style',
  severity: 'ERROR',
  text: 'Failed to load sprite: unsupported image type' }

But, if I open https://api.mapbox.com/styles/v1/mapbox/basic-v9/sprite.png?access_token=pk.xxx, the image is there.

I did some googling and found that this might be related to libsdl, so I installed the libsdl2-image-dev package, but it didn't help. Do you by any chance know what I can do to fix this problem and use sprites?

Btw.: If I remove the sprites entry from the JSON, I don't get any errors and I am able to generate the map images.

[mikemorris]

@Maarius Ack, I'm guessing that the spriteUrl is parsed differently depending on context, and hardcoding the JSON link is feeding JSON to a request that expects the PNG from https://api.mapbox.com/styles/v1/mapbox/basic-v9/sprite.png?access_token=pk.xxx - I'm not sure if there's a way to fix this in the stylesheet alone or not...

This definitely doesn't have anything to do with libsdl.

[Mariusio]

Thanks for the answer. So is there any way for me to fix this issue (e.g adjusting the style.json or my node code)? Or is it simply not possible to use sprites with mapbox-gl-native for node?

[mikemorris]

@Maarius It's something that should be possible, I'm just not too familiar with using the library in this way.

@vkurchatkin @abuchanan Were either of you able to get this working?

[Mariusio]

@mikemorris Thanks for your help, it is highly appreciated. Is there someone else who we could ping (within Mapbox maybe?) who could help to solve this issue?

[jingsam]

Hi @Maarius , I am trying to use node-mapbox-gl-native on a Ubuntu 14.04 Server VM, but can't work. Could you please provide more details on how to configure your server to get it works?

[Mariusio]

@jingsam I would suggest to not hijack this issue and rather open a new one. Please post the steps that you did and the error message that you get in there and I will try to help you!

[jingsam]

Hi @Maarius bellow is what I have done: ENV:

Ubuntu 14.04 Server VM
node 4.4.4
npm 2.15.1

1. After I installed mapbox-gl-native using npm install mapbox-gl-native --save, when I run a simple example, it complains:

libGL.so.1: cannot open shared object file: No such file or directory

So I do some search and run this:

sudo apt-get install libxi-dev libglu1-mesa-dev x11proto-randr-dev \
                     x11proto-xext-dev libxrandr-dev \
                     x11proto-xf86vidmode-dev libxxf86vm-dev \
                     libxcursor-dev libxinerama-dev

OK, it solved this error

1. Try again, anther error comes:

/usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.20' not found

That easy, install g++-5, solved!

1. Try again again, I am stuck with this:

terminate called after throwing an instance of 'std::runtime_error'
  what():  Failed to open X display.
Aborted (core dumped)

[Mariusio]

@jingsam please see https://github.com/mapbox/mapbox-gl-native/issues/4922#issuecomment-216574903

1. You have to install xvfb
2. Then run it with sudo Xvfb :10 -ac
3. Then set the ENV variables like:

export DISPLAY=:10 
export LD_LIBRARY_PATH="{PATH_TO_MESA_LIB_DIRECTORY}"

[jingsam]

OK,

 export DISPLAY=:10
➜  ~ export LD_LIBRARY_PATH=/home/tingzhang/mesa/lib
➜  ~ sudo Xvfb :10 -ac

武大 陈宁 2016/5/23 21:06:45
➜  ~ export DISPLAY=:10
➜  ~ export LD_LIBRARY_PATH=/home/tingzhang/mesa/lib
➜  ~ sudo Xvfb :10 -ac
Initializing built-in extension Generic Event Extension
Initializing built-in extension SHAPE
Initializing built-in extension MIT-SHM
Initializing built-in extension XInputExtension
Initializing built-in extension XTEST
Initializing built-in extension BIG-REQUESTS
Initializing built-in extension SYNC
Initializing built-in extension XKEYBOARD
Initializing built-in extension XC-MISC
Initializing built-in extension SECURITY
Initializing built-in extension XINERAMA
Initializing built-in extension XFIXES
Initializing built-in extension RENDER
Initializing built-in extension RANDR
Initializing built-in extension COMPOSITE
Initializing built-in extension DAMAGE
Initializing built-in extension MIT-SCREEN-SAVER
Initializing built-in extension DOUBLE-BUFFER
Initializing built-in extension RECORD
Initializing built-in extension DPMS
Initializing built-in extension Present
Initializing built-in extension DRI3
Initializing built-in extension X-Resource
Initializing built-in extension XVideo
Initializing built-in extension XVideo-MotionCompensation
Initializing built-in extension SELinux
Initializing built-in extension GLX
[dix] Could not init font path element /usr/share/fonts/X11/cyrillic, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/Type1, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi, removing from list!

[jingsam]

Would be less painful if I deploy my program on a linux with GUI?

[Mariusio]

@jingsam please make sure that you have the following packages installed: sudo apt-get install xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic

See also: http://stackoverflow.com/a/34549360

[jingsam]

Solved this problem.

➜  ~ sudo Xvfb :10 -ac
Initializing built-in extension Generic Event Extension
Initializing built-in extension SHAPE
Initializing built-in extension MIT-SHM
Initializing built-in extension XInputExtension
Initializing built-in extension XTEST
Initializing built-in extension BIG-REQUESTS
Initializing built-in extension SYNC
Initializing built-in extension XKEYBOARD
Initializing built-in extension XC-MISC
Initializing built-in extension SECURITY
Initializing built-in extension XINERAMA
Initializing built-in extension XFIXES
Initializing built-in extension RENDER
Initializing built-in extension RANDR
Initializing built-in extension COMPOSITE
Initializing built-in extension DAMAGE
Initializing built-in extension MIT-SCREEN-SAVER
Initializing built-in extension DOUBLE-BUFFER
Initializing built-in extension RECORD
Initializing built-in extension DPMS
Initializing built-in extension Present
Initializing built-in extension DRI3
Initializing built-in extension X-Resource
Initializing built-in extension XVideo
Initializing built-in extension XVideo-MotionCompensation
Initializing built-in extension SELinux
Initializing built-in extension GLX

Is Xvfb successfully running?

[mikemorris]

@jingsam @Maarius the best place to check for simply getting node-mbgl running on Linux is in https://github.com/mapbox/mapbox-gl-native/blob/6c05e51cbc0781d6826370a618dc61855b79bb4f/scripts/travis_setup.sh, which contains our configuration for running render tests on Travis CI, including configuring xvfb and Mesa.

sudo apt-get install xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic should not be necessary - the packages we install to get this running on Travis CI (Ubuntu 12.04 IIRC) are listed at https://github.com/mapbox/mapbox-gl-native/blob/6c05e51cbc0781d6826370a618dc61855b79bb4f/.travis.yml#L7-L11

@Maarius This could be a bug when using a non-mapbox:// sprite, looking at OnlineFileSource::request it's clear that SpriteImage and SpriteJSON share the same logic, but normalizeSpriteURL doesn't appear to be magically adding an extension.

What happens when you usehttps://api.mapbox.com/styles/v1/mapbox/basic-v9/sprite.png?access_token=pk.xxx (or with .json extension) instead of the URL without an extension?

[jingsam]

I have spend a whole day trying to setup a system environment to run node-mapbox-gl-native on a Ubuntu VM. I go through all instructions mentioned above. However, I failed again, it complains:

libGL error: failed to load driver: swrast

This is my second time trying to get mapbox-gl-native to work. I am very depressed now. Setting up the environment of mapbox-gl-native is painful and the document is deficient. Could you please provide more clues to help me and others to use mapbox-gl-native? I will appreciate for your help! @mikemorris

host OS: Windows 10 VM: ubuntu 14.04 LTS node: v4.4.4

[bittersweet]

Can you provide more info @jingsam ? Are you running via Xvfb? That error is familiar to me, and I had the same thing with various Xvfb incantations.

Xvfb :99 -screen 0 1024x768x8
# running my node code results in 
# libGL error: No matching fbConfigs or visuals found
# ibGL error: failed to load driver: swrast

The following copied from the mentioned travis configuration works for me as well. From memory, the x24 was important and are you setting +extension GLX?

start-stop-daemon --start --pidfile ~/xvfb.pid --make-pidfile --background --exec /usr/bin/Xvfb -- :99 -screen 0 1024x768x24 -ac +extension GLX +render -noreset
export DISPLAY=:99

See also: http://unix.stackexchange.com/questions/104914/why-does-xvfb-run-glxgears-fail-with-an-swrast-error

[jingsam]

@bittersweet thank you very much, eventually make the example works!

[jingsam]

I send a PR https://github.com/mapbox/mapbox-gl-native/pull/5632 to fix the example @mikemorris

[mikemorris]

libGL error: failed to load driver: swrast

This is a Mesa error from not being able to find the software rasterizer (as opposed to hardware GPU drivers) - I've hit it before when building Mesa from source without the required drivers, but I suppose it's possible to hit it via configuration issues too - Mesa can be a bit finnicky. Thanks for helping out @bittersweet!

Can we close out this issue now, or are there still unresolved issues pertaining to the original post? I'll follow up on the PR and get that working example merged, thanks @jingsam!

[Mariusio]

@mikemorris I would still like to test your recommendation from this comment: https://github.com/mapbox/mapbox-gl-native/issues/4922#issuecomment-225724657, then we can close. Or should I rather open a new issue if I can't get it to work?

[mikemorris]

@Maarius Ah, yea, let's open a new issue specifically for tracking the SpriteJSON/SpriteImage logic, as that's pretty far afield from the original topic here.

[Mariusio]

Ok sure, thanks for all the help so far already!

[zuo1188]

default setting of xvfb have have problems when running mapbox-gl-native, screen depth shoud be 24 ,so run nodejs with : xvfb-run --auto-servernum -s "-screen 0 640x480x24" {your app}