Open zdila opened 5 years ago
Note that I had no such issue with debian linux running in docker.
I've checked all threads of node process. All are waiting on futex
/FUTEX_WAIT_PRIVATE
but one was epoll_pwait(13,
.
Some output from gdb:
(gdb) bt
#0 0x00007f8bfa29cfd1 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fff49699508 in ?? ()
#2 0x00007f8bf6cd70cc in mapnik::CreateStatic<mapnik::datasource_cache>::create()::static_memory () from /freemap-mapnik/node_modules/mapnik/lib/binding/lib/libmapnik.so
#3 0xffffffff80000010 in ?? ()
#4 0x00007f8bfa29c561 in __timedwait_cp () from /lib/ld-musl-x86_64.so.1
#5 0x0000000000000000 in ?? ()
datasource_cache
is the only place in Mapnik where std::recursive_mutex
is used: src/datasource_cache.cpp#L163. I'm wondering whether this cannot be somehow connected to your problem. Maybe the mutex actually created is not recursive one.
but one was
epoll_pwait(13,
You can take a look what is behind the file descriptor 13
by ls -l /proc/6482/fd
with pid of your app.
It is anon_inode:[eventpoll]
.
I can prepare a testcase if it would help.
Are you building Mapnik directly in Docker? In that case it would be great if you can share the dockerfile.
Dockerfile:
RUN apk add --update nodejs npm gcompat
RUN mkdir /test
WORKDIR /test
RUN npm init -y
RUN npm i --save mapnik
RUN echo "require('mapnik').register_default_input_plugins();" > index.js
Steps:
docker build -t test .
docker run -it -w /test test node .
docker run
should terminate but it doesn't.
Thanks, I can reproduce it. Will try to take a look deeper later today.
I still don't know what exactly causes that deadlock, but Mapnik package from Alpine works - no surprise - well:
My test app:
#include <mapnik/datasource_cache.hpp>
#include <iostream>
#undef NDEBUG
#include <cassert>
int main()
{
auto & cache = mapnik::datasource_cache::instance();
assert(cache.register_datasources("/usr/lib/mapnik/input/", true));
for (auto const & name : cache.plugin_names())
{
std::clog << name << std::endl;
}
return 0;
}
Output:
$ ./test
csv
gdal
geojson
ogr
pgraster
postgis
raster
shape
sqlite
topojson
I would maybe just not use the prebuilt Mapnik from npm, rather build it from latest sources inside Docker. It is also way how to deploy patches quickly. To build Mapnik itself is simple, I would like to try also node-mapnik:
FROM alpine:latest
RUN echo "http://repository.fit.cvut.cz/mirrors/alpine/edge/main" > /etc/apk/repositories; \
echo "http://repository.fit.cvut.cz/mirrors/alpine/edge/community" >> /etc/apk/repositories; \
echo "http://repository.fit.cvut.cz/mirrors/alpine/edge/testing" >> /etc/apk/repositories
RUN apk update && apk upgrade
RUN apk add bash make git python gdal-dev cairo-dev \
postgresql-dev harfbuzz-dev g++ boost-dev sqlite-dev \
proj4-dev jpeg-dev tiff-dev libwebp-dev
RUN mkdir /build
WORKDIR /build
RUN git clone https://github.com/mapnik/mapnik.git
WORKDIR mapnik
RUN git checkout v3.0.x
RUN git submodule update --init
RUN ./configure
RUN JOBS=2 make
RUN make install
Building node-mapnik:
FROM mapy-sk-mapnik-build:latest
RUN apk add nodejs npm
WORKDIR /build
RUN git clone https://github.com/mapnik/node-mapnik.git
WORKDIR node-mapnik
RUN git checkout v3.0.x
RUN git submodule update --init
RUN make release_base
Test image:
FROM mapy-sk-mapnik-build-node:latest
RUN mkdir /test
WORKDIR /test
COPY package.json .
RUN npm i
RUN echo "var mapnik = require('mapnik'); mapnik.register_default_input_plugins();console.log(mapnik.datasources());" > index.js
package.json:
{
"name": "test",
"version": "1.0.0",
"description": "",
"dependencies": {
"mapnik": "file:/build/node-mapnik"
},
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "ISC"
}
Output:
$ docker run --rm -i -t mapy-sk-mapnik-build-node-test:latest node .
[ 'csv',
'gdal',
'geojson',
'ogr',
'pgraster',
'postgis',
'raster',
'shape',
'sqlite',
'topojson' ]
Great, thanks!
Anyway we already use debian-based containers but we'll definitively use your instructions if we decide to switch to alpine :-)
Not sure whether I should close this issue now. I'll leave it up on you.
Call
mapnik.register_default_input_plugins()
never returns.Last lines from strace output:
Removing
/freemap-mapnik/node_modules/mapnik/lib/binding/lib/mapnik/input/csv.input
makes it halt on another*.input
file. If I remove all files from/freemap-mapnik/node_modules/mapnik/lib/binding/lib/mapnik/input/
then the call returns but obviously it fails later as necessary drivers are missing.