open-streets-dc won't parse

springmeyer commented 13 years ago

d:carto-parser dane$ ./carto-parser ~/Documents/MapBox/project/open-streets-dc/style.mss 
Error: Parser failed!

./carto-parser ~/Documents/MapBox/project/open-streets-dc/labels.mss 
<?xml version="1.0" encoding="utf-8"?>
<Map srs="+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs"/>

d:carto-parser dane$ ./carto-parser ~/Documents/MapBox/project/open-streets-dc/highway.mss 
Error: Parser failed!

springmeyer commented 13 years ago

any status on this? I'd love to know how far this is out from working, so I can assess whether/how to involve other tilemill devs in testing.

springmeyer commented 13 years ago

ping @rundel.

/cc @rcoup

rundel commented 13 years ago

I am working on it currently which involves getting attribute expressions working which is now added but has broken a couple other things. Once I have that sorted there are a few other bits to fix with regard to style names.

There are a couple questions I had with the name decorators, I had assumed that # was used exclusively to identify a datasource and . was for subrules or mixins which does not appear to be the case with this stylefile. Is there any clarification between these two?

springmeyer commented 13 years ago

yes, either .class or #id can be used (as datasources can be labeled with a .class optionally), or they can be mixed which @ajashton does in that style - screenshot.

I'm realizing now the docs on this are non-existant. But, I found @ajashton explaining the logic here: https://github.com/mapbox/carto/issues/74 and I'll ping him to help explain further if you need.

rundel commented 13 years ago

That helps clarify things nicely, makes things a bit more complicated to work out but it is good to know how things are supposed to work

springmeyer commented 13 years ago

awesome, looking forward. any more questions, post here when they come up.

rundel commented 13 years ago

The style now parses and processes but results in a segfault somewhere in the xml serialization (probably because I've screwed something up in the map class somewhere). Probably some other issues I havent notice yet due to the size of the styles but we're much closer.

I've also added the open-street-dc example into tests with the minor change of adding type attributes to the datasource since mapnik was complaining.

For the time being run the parser inside the same directory as the stylefiles since relative paths stuff needs some work.

rundel commented 13 years ago

I finally tracked down the segfault issue in mapnik, simple issue with marker symbolizers not having default values for their type and placement enums. Following patch fixes things. I am able to parse and get an almost working result back, there is still the remaining issue of zoom attributes leftover in the filter which I am working on, but other than that everything is parsing and processing, whether it is correct or not remains to be seen.

diff --git a/src/markers_symbolizer.cpp b/src/markers_symbolizer.cpp
index 5ab9472..7588d66 100644
--- a/src/markers_symbolizer.cpp
+++ b/src/markers_symbolizer.cpp
@@ -53,8 +53,8 @@ markers_symbolizer::markers_symbolizer()
       width_(5.0),
       height_(5.0),
       stroke_(),
-      marker_p_(),
-      marker_type_() {}
+      marker_p_(MARKER_LINE_PLACEMENT),
+      marker_type_(ARROW) {}

 markers_symbolizer::markers_symbolizer(path_expression_ptr filename) 
     : symbolizer_with_image(filename), 
@@ -66,8 +66,8 @@ markers_symbolizer::markers_symbolizer(path_expression_ptr filename)
       width_(5.0),
       height_(5.0),
       stroke_(),
-      marker_p_(),
-      marker_type_() {}
+      marker_p_(MARKER_LINE_PLACEMENT),
+      marker_type_(ARROW) {}

 markers_symbolizer::markers_symbolizer(markers_symbolizer const& rhs) 
     : symbolizer_with_image(rhs),

springmeyer commented 13 years ago

great, made that fix upstream in svn just now.

I'll try to take a closer look at the output XML later today, but right off I'm seeing some debug crud at the top of the xml:

#processed_p #processed_p
#shoreline_300 #shoreline_300
.highway .highway
.highway-label .highway-label
.highway.fill .highway
.highway.fill-oneway_arrow .highway
.highway.line .highway
.location .location
.natural .natural
.water .water
<?xml version="1.0" encoding="utf-8"?>
<Map srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over" background-color="rgb(255,255,255)">
    <FontSet name="2bfe0e7995407ee6">
        <Font face-name="Ubuntu Regular"/>
        <Font face-name="Arial Regular"/>
        <Font face-name="DejaVu Sans Book"/>
    </FontSet>
    <Style name="#processed_p">
        <Rule>
            <Filter>([zoom]&gt;=11)</Filter>
            <PolygonSymbolizer fill="rgb(255,255,255)"/>
        </Rule>
    </Style>

rundel commented 13 years ago

I thought I had gotten rid of that debug messaging, lastest revision should be clean.

springmeyer commented 13 years ago

yep, latest is clean, sorry, see that now.

rundel commented 13 years ago

Added the zoom level stuff, let me know what the output for this looks like now or if there are still some output issues.

springmeyer commented 13 years ago

Output issues I see now include:

'highway' style is totally empty - not sure why - this is what carto.js compiled looks like: https://gist.github.com/1177890
we need filter mode added: https://github.com/rundel/carto-parser/issues/18
sorting of rules is slightly different (might be a carto.js but, we'll have to dig more)
just comparing the 'highway-label' style it appears carto.js output has a few MinScaleDenominators, carto c++ has none

springmeyer commented 13 years ago

btw, here is what the carto-parser c++ outputs for me (that I am comparing too): https://gist.github.com/1177898

rundel commented 13 years ago

Updates have gotten some things showing up, parks + water + city label. Working on the highways to see what is up.

Currently the city label shows up the wrong size because of https://github.com/rundel/carto-parser/issues/19 which probably affects lots of other bits too

springmeyer commented 13 years ago

great work, glad things are starting to show up for you. On my end (clang trunk, osx 10.6) I'm now getting a blank map:


$ ./carto-parser tests/open-streets-dc/open-streets-dc.mml 
Error: Unknown keyword: bounds at Line: 7 Col: 3
Error: Unknown keyword: center at Line: 12 Col: 3
Error: Unknown keyword: format at Line: 13 Col: 17
Error: Unknown keyword: interactivity at Line: 14 Col: 24
Error: Unknown keyword: minzoom at Line: 15 Col: 14
Error: Unknown keyword: maxzoom at Line: 16 Col: 15
Error: Unknown keyword: class at Line: 108 Col: 3
Error: Unknown keyword: legend at Line: 109 Col: 476
Error: Unknown keyword: name at Line: 110 Col: 28
Error: Unknown keyword: description at Line: 111 Col: 57
Error: Unknown keyword: attribution at Line: 113 Col: 0
<?xml version="1.0" encoding="utf-8"?>
<Map srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over"/>

springmeyer commented 13 years ago

ha, nm! was using an old binary I build, not 'example'. So, yes! I'm now seeing stuff:

carto c++:

carto.js:

rundel commented 13 years ago

A bit more progress, my logic for matching styles with layers based on id and class attributes appears to be wrong. Any explanation on how carto matches class attributes to styles? I haven't seen the following before:

    {
      "id": "highway-outline",
      "name": "highway-outline",
      "srs": "",
      "class": "highway line",
      "Datasource": {
        "file": "layers/dc_osm_highway/dc_osm_highway.shp",
        "id": "highway-outline",
        "srs": "",
        "project": "open-streets-dc",
        "type": "shape"
      },
      "geometry": "linestring"
    },
    {
      "id": "highway",
      "name": "highway",
      "srs": "",
      "class": "highway fill",
      "Datasource": {
        "file": "layers/dc_osm_highway/dc_osm_highway.shp",
        "id": "highway",
        "srs": "",
        "project": "open-streets-dc",
        "type": "shape"
      },
      "geometry": "linestring"
    }

springmeyer commented 13 years ago

@ajashton - can you give a summary of how the matching works? @rundel - here is what carto.js outputs for XML: https://gist.github.com/1177890

ajashton commented 13 years ago

The id/class matching is meant to work just like CSS - for a layer to match a style the layer must have the same ID of the style selector (if any) and all of the classes of the selector. The layer may have more classes than the selector, but not fewer.

Eg, for #layer.foo.bar.bat:

#layer { /* matches */ }
.bar { /* matches */ }
.foo.bat { /* matches */ }
.foo.boo { /* does not match */ }
#layer.boo { /* does not match */ }

This is useful for styling, eg for the following layers:

stat1.stat.low
stat2.stat.low
stat3.stat.high

.stat {
    polygon-fill:red; /* matches #stat1, #stat2, & #stat3 */
    .low { polygon-opacity:0.5; } /* matches #stat1 & #stat2 */
    .high { polygon-opacity:0.75; } /* matches #stat3 */
 }

rundel commented 13 years ago

Some progress, thanks to the info I now have matching working for everything but attachments currently which gets us a little closer to something that looks correct:

carto c++

springmeyer commented 13 years ago

woot!

springmeyer commented 12 years ago

trying this again now and I get:

~/projects/carto-parser[master]$ ./carto ~/Documents/MapBox/project/open-streets-dc/project.mml 
Assertion failed: (false), function operator(), file include/utility/utree.hpp, line 32.
Abort trap: 6

rundel / carto-parser

open-streets-dc won't parse #9

stat1.stat.low

stat2.stat.low

stat3.stat.high