use JOSM mapcss validators to define tag validations

mikelmaron commented 8 years ago

JOSM validation tests are defined here https://josm.openstreetmap.de/browser/josm/trunk/data/validator?order=name

Investigate parsing this format in node, and executing the checks from TileReduce. The cli for this processor would take an additional argument specifying the mapcss file.

mikelmaron commented 8 years ago

To work the mapcss files in osmlint, they need to be parsed by and rules implemented in javascript. mapcss files have a funky, custom format. Here's a start of investigating this format, and recommended approach to use these files in osmlint.

The mapcss parser and execution is of course present in JOSM, but takes a lot of digging to find within a lot of Java (thanks to @aarthy for helping to investigate), but honestly does not reveal very much that is helpful work with these files in Javascript.

Straightforward tag matching examples

The mapcss file format broadly consists of a series of regular expression encoded rules on the feature type (node/way/relation), followed by conditions on the tags. When a rule condition is met, there's a warning string and classification level. There are also tests defined directly within each rule.

For example, from religion.mapcss

*[religion =~ /^(christian|jewish|muslim)$/][!denomination] {
      throwOther: tr("religion without denomination");
      assertMatch: "node religion=christian";
      assertNoMatch: "node religion=christian denomination=catholic";
      assertNoMatch: "node religion=foobar";
    }

Tests on all feature types (*), if the religion tag matches values in the regular expression ^(christian|jewish|muslim)$, and denomination is not set, !denomination, it throws an "other" level alert of religion without denomination.

And from highway.mapcss

node[highway =~ /motorway|trunk|primary|secondary|tertiary|unclassified|residential|service|living_street|pedestrian|track|path|footway|cycleway|bus_guideway|bridleway/]
[highway!=motorway_junction][highway!=services] {
      throwWarning: tr("wrong highway tag on a node");
...
    }

Any node with a highway tag would throw a "warning" level alert, if it had any value except motorway_junction or services.

More complicated functionality we can avoid

Those above are the simple cases we'd like to work with. There are additional complexities to avoid, like

the assignment and use of variables within mapcss:

way[highway=~/^(motorway|trunk|primary|secondary|tertiary)$/] {
      set major_road;
}

way.major_road[!ref] {
      throwOther: tr("highway without a reference");
      assertMatch: "way highway=primary";
      assertNoMatch: "way highway=primary ref=123";
}

references to geometric functions:

area:closed:areaStyle ⧉ area:closed:areaStyle {
      throwOther: tr("Overlapping Areas");
}

An Approach to MapCSS for Tag Valiation in osmlint?

There's a large body of work encoded in the mapcss rules used in JOSM, and they continue to be updated. There is value in working with this peculiar format, to take advantage of that knowledge.

Recommend approach is to only utilize the simplest tag rules, and filter out the rest. The file parser (perhaps using something like https://www.npmjs.com/package/simple-text-parser) needs to build an encoding of the rules in javascript. Each rule could take the form of an array of strings, each a rule condition, mapped to the warning string and warning level if all are matched.

To apply each rule, make a string substitution on the rule to bring in the appropriate features.properties variable to perform the regex against, and eval.

mikelmaron commented 8 years ago

@Rub21 @geohacker ^^

Rub21 commented 8 years ago

@mikelmaron : Some progress here:

I create a equivalent mapcss validator in JavaScript

e.g : mapcss --> Javascrpit

I've added feature_type: , match: throw: as values and a assess function which is going to filter the objects. acording the values.

the equivalent for filter the objects are:

e.g :
*[religion =~ /^(christian|jewish|muslim)$/][!denomination] = this.feature_type && this.match[val.properties.religion] && !val.properties.denomination

e.g : *[religion=christian][denomination][denomination !~ /^(anglican|apostolic|...] = this.feature_type && this.match[val.properties.religion] && !this.nomatch[val.properties.denomination]

the validator is really faster, here the result.

I haven't used the file parser, because it made some illogical parser of mapcss, and I notice the parser file is not good to use in map.js

The code of religion validator is in branch mapcss.

osmlab / osmlint