barrier=retaining_wall as closed way not added to polygon table

[mboeringa]

Hi,

Another issue I discovered looking at my renderer results: barrier=retaining_wall as closed ways is not being added to the polygon table. I do see the same feature rendered as line in my render results, but not as closed polygon surface. I checked the below linked feature in JOSM, and it seems and validates fine there:

http://www.openstreetmap.org/way/229579520

This may affect multipolygon relations with the same tag as well.

[mboeringa]

Just realized the feature does not have an area=yes/no tag, which it probably should, as defaulting to line seems obvious for something tagged as "wall", even though clearly this example above is intended as polygon...

The typical OSM, and more general, cartography related dilemma's: how to represent an object in the real world in a GIS?

On the other hand: it does seem to render as polygon in OSM Standard, so should we keep this similar?

[ThomasEmge]

Marco, this is similar to the issue with the national parks. With the code changes in place, it will be a polygon.

It is a little unfortunate that OSM doesn't have an explicit polygon type. There can be hints like the area=yes tag but attribution is optional. Our code will try to read the 'hints' but it might just end up having to 'walk' the geometry to detect if the way is closed or not.

[mboeringa]

Marco, this is similar to the issue with the national parks. With the code changes in place, it will be a polygon.

It is a little unfortunate that OSM doesn't have an explicit polygon type. There can be hints like the area=yes tag but attribution is optional. Our code will try to read the 'hints' but it might just end up having to 'walk' the geometry to detect if the way is closed or not.

I am fully aware of this limitation, and others like the problems with tags put on outer ways instead of multipolygon relations, potential hybrid relations (point, line, polygon in one relation, fortunately not to common), and the issues this represents for usage in any GIS, whether ArcGIS, QQIS, AutoCAD Map or whatever.

In general though, I think the Load OSM File tool does a pretty decent job (and that is an understatement), especially in the light of only one other similar tool being out there: osm2pgsql, and that tool seemingly being even for the most intensely involved in its maintenance, still part "black magic", if I consider some of the discussion I have seen going on...

[mboeringa]

By the way,

In case an area=no tag is added to an object, you do direct the features to the line table, even if a closed way (not multipolygon relation, because these should ignore area tagging)?

[ThomasEmge]

Correct, I am interpreting the tags as explicit information. If it is closed but the tag is area=no, then it will become a line. The only exception is the highway tag, I ran into the issue with roundabouts ;)

Undetermined ways end up in the line feature class, undetermined relations end up in the relation table.

One major reason for the recent code changes were issues we discovered with respect to the ebola support. A number of the political boundaries were expressed as 'mixed' relations, and the 'mix' coming from ways and a label point. The code is now ignoring label points when trying to figure out the underlying geometry.

[mboeringa]

One major reason for the recent code changes were issues we discovered with respect to the ebola support. A number of the political boundaries were expressed as 'mixed' relations, and the 'mix' coming from ways and a label point. The code is now ignoring label points when trying to figure out the underlying geometry.

Good you mention this, because I now discovered the same issue with some of the Australian states. The relations are not being added to the polygon table, due to the label point issue you describe. See the link to the label point of Queensland, and the link from there to the hybrid relation of Queensland with its actual polygon (2316595): http://www.openstreetmap.org/node/1971329414#map=5/-22.370/145.679&layers=H

[mboeringa]

A number of the political boundaries were expressed as 'mixed' relations, and the 'mix' coming from ways and a label point. The code is now ignoring label points when trying to figure out the underlying geometry.

Thomas,

You are not only ignoring nodes marked as "label" in your code? Your code change should in fact ignore any node combined with ways in a multipolygon relation.

For example, although this is probably bad practice and JOSM flags a warning for it during validation, the gigantic castle "Schloss Nymphenburg" in Munchen, Germany, has a node added as a child member to the multipolygon relation. The node contains the name of the castle (in several languages). These loose nodes added to a multipolygon relation as members should all be ignored during polygon creation. See the link to the relation below:

https://www.openstreetmap.org/relation/2144986#map=17/48.15830/11.50439

I discovered this particular example because in my map of Munchen, Schloss Nymphenburg was completely missing... The main OSM website, does display it though. See the attached image (right is from my OSM Renderer for ArcGIS). Also notice the missing forest areas in my rendering, probably due to another problem: multipolygons having tags only on outer ways, not on the relation itself:

[ThomasEmge]

No, the code is not limited to 'label' or 'name' tags. You really need to try the current dev version on what has been resolved and what is still problematic. Nymphenburg is an interesting example as it imports ok, but the 'cuts' are not visible. I'll check on the issue updating the supportingElement attributes.

[mboeringa]

but the 'cuts' are not visible

You mean the inner courts of the castle, that should be donut holes in the building complex?

[ThomasEmge]

Correct, they are there but the actual polygons are still there as well. The reason there is because the individual building polygons all have different attributes.

https://www.openstreetmap.org/api/0.6/relation/2144986/full

Any suggestions?

[mboeringa]

Have you read the link I posted to the osm2pgsql Github repository in the other issue I opened here (https://github.com/Esri/arcgis-osm-editor/issues/65), and especially the remarks about the "superseded" option that seems to be part of osm2pqsql? I am not entirely sure what it is or how it is implemented, but it may be interesting read for you...

[ThomasEmge]

Yes, reading through this issue I would say that we have a similar approach to the pqsql flagging with the supportingElement attribute. So far the implemented attribute handling looked good until you discovered the Nymphenburg case. Which is an interesting case as the building parts have different tags to describe the uniqueness of the castle. If the building parts didn't have any tags they would be flagged as supporting elements and only the aggregated geometry of the castle would be there. 'There' being the difference between supportingElement 'yes' and 'no'. Now we have both geometries.

Honestly I don't know how I should handle this situation and I need some suggestions from our users and OSM experts like yourself.

[mboeringa]

I wouldn't qualify myself as an "OSM expert", I was never really deeply involved in the true framework, like Ronald Olbricht with his Overpass API to name but one, but admittedly, over the course of the last year, I have read more OSM Wikipedia pages, developed render rules, and fine tuned them based on real-world rendering results, than most people will in a life time...

Anyway, that is all a bit off-topic. I will look into some good examples for the other issue I posted, and think a bit more about some possible changes.

I will try downloading the latest development build as well...

[ThomasEmge]

I'll finish up some other code changes and then test with a new exception for "building:part" tags. Based on taginfo it seems to be used quite frequently.

[mboeringa]

Is there any way I could get a compiled - ready to use - *.tbx toolbox file of the latest version...?

[ThomasEmge]

I could provide you with updated dlls that you can just drop in place. Updates to the tbx file will happen after the dlls are done, and even then there will only be 'minor' changes. I'll get in touch with you once the dlls are complete.

[mboeringa]

Ok, thanks, that would be great for further testing.

[mboeringa]

I could provide you with updated dlls that you can just drop in place. Updates to the tbx file will happen after the dlls are done, and even then there will only be 'minor' changes. I'll get in touch with you once the dlls are complete.

Thomas, these updated dlls are definitely an improvement.

I still need to do further testing, involving changes to my OSM Renderer for ArcGIS models to revert "workarounds" I added to deal with some of the polygon issues, but this does seem to make a difference. Look at the below images. The first the original one (right part of the image), the second below that the new one. Schloss Nymphenburg is now properly dealt with, and the forested areas on the left that were completely missing with the old data loader are now visible.

However, the tool now also highlights the one major remaining issue I think we need to deal with. Look at the missing "grass" around "Amalienburg" in the lower left corner, This is actually a properly formed Multipolygon relation of forest with donut holes for the non-wooded areas, however the multipolygon itself does not have any relevant tag, hence it is replaced by its outer way. This is exactly the issue I posted:

"Option needed to automatically move tags from outer way to multipolygon relation (Load OSM File tool)" #65

I really think there should be a user-selectable option to move the tags of outer ways to the multipolygon, as per the conditions I wrote in that issue. That would resolve this problem, and allowing a rendering nearly fully equivalent to osm2pgsql.

The particular multipolygon of this wooded area, it is this one:

http://www.openstreetmap.org/relation/335052

could actually serve as a test case for you, as it is about the most simple form that I was talking about: just one outer way, so no potential issues with conflicting tags caused by diverse tagging on multiple outer ways.

I'd love to see an improved version of the dlls capable of dealing with this...

Marco

[ThomasEmge]

Marco, yes indeed that was a bug and the attributes of the outer ring are now appended to the relation itself. However, it doesn't resolve the issue completely.

The 'hole' in the forest area happens to have the following attributes: <tag v="park" k="leisure"/> <tag v="Kronprinzengarten" k="name"/> and as such (or in the current version of the software) has the enough standing power to be on its own. The result is that the attribute supportingElement is set to no. It just so happens that it (the hole) is tagged as a special area in the surrounding forest. As a result, the hole does exist within the forest but is also overlapped but the feature describing the leisure park.

[mboeringa]

However, it doesn't resolve the issue completely.

It just so happens that it (the hole) is tagged as a special area in the surrounding forest. As a result, the hole does exist within the forest but is also overlapped but the feature describing the leisure park.

I don't quite understand what you are trying to say here. This is not an issue, this is the expected behaviour, and how osm2pgsql probably treads this.

Any "inner" members of a multipolygon relation with tags other than "inner", like leisue=park, should indeed be a "feature-on-its-own", and be added to the polygon table with osmSupportingElement=no to be visible.

However, these features shouldn't be added twice to the polygon table, 1) because they are a feature in its own right, and 2) because they are a member of a larger multipolygon relation. We don't want to render the same feature multiple times just because of relationship membership. Potentially, there could be an unlimited number of relationship memberships for a single closed way with a certain tag set...

I will try out the new dlls you send and see how it end up in the rendering.

[mboeringa]

Maybe the confusion is because I referred to JOSM showing donut holes for the wooded area. JOSM seems to only display a kind of transparent fill for certain features. I am not sure what exactly the rules for this are: just a closed way / valid multipolygon, or certain features only as defined by some configuration in JOSM.

This specific drawing behaviour of JOSM doesn't however imply a relevant inner member, shouldn't be a feature on its own, it is just a way of displaying the data, nothing more.

In fact, the leisure=park inner way of the wooded area, if you look well at the screenshot I posted in the other issue thread, is rendered with a fill too in JOSM.

[ThomasEmge]

The challenge arises from the data model OSM uses and what ArcGIS uses. In OSM terms relations are a mixed bag. A potential mixture of points and ways that are topologically organized. ArcGIS has geometries of points, lines and polygons (and each can be multi-part) but a feature can have only one geometry (and one geometry type) and one set of attributes. If the inner way (the leisure park) of the forest has attributes and it is assumed to be its own feature then it needs to have its own geometry and be a stand-alone feature. ArcGIS can't assemble its geometries through relationships but needs to represent them explicitly. That is the reason why you get two or more features and geometries: One for the forest and one for the park.

[mboeringa]

ArcGIS can't assembled its geometries through relationships but needs to represent them explicitly.

I am fully aware of this and the other stuff you wrote. I was just trying to say that the osm2pgsql / Mapnik / OpenStreetMap Carto render pipeline, almost certainly uses the same strategies as you have implemented, that is, multiple database records of potentially different geometry types (point, line, polygon) to - kind of - rudimentary represent a single OSM relationship.

That is the reason why you get two or more features and geometries: One for the forest and one for the park.

I understand, and this is in fact the desired behaviour, and what I need for my renderer to be able to mimic the osm2pgsql / Mapnik / OpenStreetMap Carto render pipeline.