No way to compile to XML with latest Mapnik and Carto

[yohanboniface]

Certainly linked to mapbox/carto#20

So we need to try to narrow the "guilty" changes in the mapnik/carto stack, and possibility workaround mapbox/carto#20 in poi.mss and road.mss because this style anyway has a too much verbose XML

[yohanboniface]

Quick git bisect point to https://github.com/mapbox/carto/commit/09cb77c53e9835a938a75c10f5dd8a8935c3d431

[yohanboniface]

BTW, workaround is to use carto 0.14.1 for now.

[yohanboniface]

Refactored to workaround mapbox/carto#20

[yohanboniface]

@skorasaurus do you want to have a look at the refactor before I deploy?

It's basically a refactor for better carto processing, but I've taken the opportunity to tackle a road issue that was here from the beginning: clashes between road casing of different colours. See some before/after:

So now main paved road does not have casing anymore, while small roads and unpaved main roads still have (basically, casing creates those artifacts when casing have different colours).

Also, the XML is now around 18k lines, while it was 227k before (yeah…).

Oh, and also: I've switched to yml instead of mml. I'll add a script for the conversion for people still using Tilemill, but it's three lines of python or nodeJS I'll put in the README in the meantime ;)

[skorasaurus]

Yes I am very interested to do so, although I won't have be able to until tomorrow. On Nov 11, 2015 10:02 AM, "Yohan Boniface" notifications@github.com wrote:

@skorasaurus https://github.com/skorasaurus do you want to have a look at the refactor before I deploy?

It's basically a refactor for better carto processing, but I've taken the opportunity to tackle a road issue that was here from the beginning: clashes between road casing of different colours. See some before/after:

[image: screenshot from 2015-11-11 15-58-24] https://cloud.githubusercontent.com/assets/146023/11093909/14eded4a-888d-11e5-89e1-e18ff1a43147.png [image: screenshot from 2015-11-11 15-58-18] https://cloud.githubusercontent.com/assets/146023/11093910/14efbf4e-888d-11e5-9852-987820bb3d8d.png

[image: screenshot from 2015-11-11 15-56-04] https://cloud.githubusercontent.com/assets/146023/11093844/c1ab161c-888c-11e5-9424-8a30ed39ae3f.png [image: screenshot from 2015-11-11 15-55-14] https://cloud.githubusercontent.com/assets/146023/11093846/c1afedfe-888c-11e5-93aa-f4e91f2d4f8d.png [image: screenshot from 2015-11-11 15-53-43] https://cloud.githubusercontent.com/assets/146023/11093842/c1a974c4-888c-11e5-8e0c-a6a421891e93.png [image: screenshot from 2015-11-11 15-53-01] https://cloud.githubusercontent.com/assets/146023/11093841/c1a6c8be-888c-11e5-9ba7-065e038fc4a1.png [image: screenshot from 2015-11-11 15-51-36] https://cloud.githubusercontent.com/assets/146023/11093843/c1aaa8e4-888c-11e5-83c6-14484b58584a.png [image: screenshot from 2015-11-11 15-50-23] https://cloud.githubusercontent.com/assets/146023/11093845/c1abca44-888c-11e5-8ef4-814306bb399b.png

So now main paved road does not have casing anymore, while small roads and unpaved main roads still have (basically, casing creates those artifacts when casing have different colours).

Also, the XML is now around 18k lines, while it was 227k before (yeah…).

Oh, and also: I've switched to yml instead of mml. I'll add a script for the conversion for people still using Tilemill, but it's three lines of python or nodeJS I'll put in the README in the meantime ;)

— Reply to this email directly or view it on GitHub https://github.com/hotosm/HDM-CartoCSS/issues/277#issuecomment-155805427 .

[pnorman]

For reference

 time ~/osm/openstreetmap-carto/repo/node_modules/.bin/carto project.mml | wc -l
   18297

real    0m1.380s

Old values were 227058 and 8.661s, both tests with carto 0.14.1

The reduced number of Mapnik rules should help rendering performance.

[yohanboniface]

@skorasaurus any feedback? :)

(My question is: should I proceed with deploying :) )

[skorasaurus]

not yet.

I'm seeing a lot of POIs not being rendered, images forthcoming.

[skorasaurus]

Left is the online server, right is my local , compiled from https://github.com/hotosm/HDM-CartoCSS/commit/2d62518dd952e49fc22dd287fee40d90dc98e72e

[pnorman]

Left is the online server, right is my local , compiled from 2d62518

You need to make sure to be using the same Mapnik version and metatile settings on both to have a valid comparison. This looks like it might be related to label clipping settings. How does it look when you display MT edges?

[skorasaurus]

@pnorman

MT edges are enabled in kosmtik (I'm using the master), by the following , right?

[skorasaurus]

I believe kosmtik master is requiring 3.4.7 (https://github.com/kosmtik/kosmtik/commit/982de02ae77af79e2f56c9aee8beea57bfab3f6c) ; but my terminal reported

[Core] Generating Mapnik XML…
[Core] Using mapnik version 3.0.0
[Core] Loading map…
[Core] Map ready

[pnorman]

believe kosmtik master is requiring 3.4.7 (kosmtik/kosmtik@982de02)

Kosmtik doesn't require mapnik, it requires node-mapnik which has its own version numbers.

Can you compare the current style version and the old style version with both in Kosmtik?

[yohanboniface]

@skorasaurus about the missing POIs, are you sure your data is up to date? I've loaded the Senegal DB right now, and I've same POIs in local than the live server:

[skorasaurus]

hmm... The data is from an extract just a week ago... I think it must be something on my end because I compared the online layout (to the left) with https://github.com/hotosm/HDM-CartoCSS/commit/21715c4a178562a96772531eef688c38c9dfe39a on the right and POIs are still missing in the right.

[skorasaurus]

Just to confirm @yohanboniface - The problem is something on my end and database. I don't know why but some data is not imported into my osm2pgsql database.

I created an extract of senegal from the africa pbf ~/prg/osmosis/bin/osmosis --read-pbf file=africa-2015-11-19.osm.pbf --bounding-polygon file=senegal.poly --used-node --write-pbf senegal.osm.pbf then imported the data into database (I have confirmed hstore and postgis are enabled, I'm using postgis 2.1.4) osm2pgsql -G -U skors -d hdmsene senegal.osm.pbf --hstore --create

The version of osm2pgsql that I have been using is 0.85.0 (a deb built at https://launchpad.net/~kakrueger/+archive/ubuntu/openstreetmap?field.series_filter=trusty )

[skorasaurus]

@yohanboniface I finally got it up and running. Bravo, you made some great changes. I love it. I explored a bit in Senegal and didn't find anything to hold you back from deploying :)

[yohanboniface]

Great news! I'll update the server as soon as I am back to Paris (this week) :)

[yohanboniface]

Deployed :)

[skorasaurus]

\o/