search

Esri / arcgis-osm-editor

ArcGIS Editor for OpenStreetMap is a toolset for GIS users to access and contribute to OpenStreetMap through their Desktop or Server environment.
Apache License 2.0
395 stars 129 forks source link

Issue with Multipolygon processing in latest build OSM File Loader. #150

Closed mboeringa closed 8 years ago

mboeringa commented 8 years ago

Hi @ThomasEmge ,

Now I have rendered larger areas, I noticed the latest build of the OSM File Loader (Load only) tool seems to have issues with the multipolygon processing. See the images below.

It slightly reminds me of one of the previous issues that seemed related to indexes. I do see the outer ways as geometries in the Polygon feature class, but not the multipolygons.

In the first image, a section of the Elbe river in Dresden is obviously missing (http://www.openstreetmap.org/relation/2335808#map=17/51.05472/13.74345). It is a multipolygon, because the permanently moored boat in the lower left section of the polygon is used as "inner" ("Theaterkahn Dresden"). However, if you compare it with the OpenStreetMap website rendering, it becomes clear that almost all multipolygons are missing here, all big buildings defined as mps in the city centre are gone.

dresden_missing_riverbank

dresden_osm_reference

I do see some multipolygons in other maps, so its not they are completely gone, but most are.

I also verified the same issue occurs in with Load OSM File tool. See the image below. I only used a subset of the render rules here related to water and infrastructure, however, you can again clearly see the section of the Elbe river missing.

dresden_load_osm_file_tool

ThomasEmge commented 8 years ago

This might be hard to pinpoint. I downloaded this area of downtown Dresden and loaded it with the 'old' and 'new' loading tool. I do have your missing polygon (the piece of the Elbe) with the 'multipolygon' and 'riverbank' tags. I also see https://www.openstreetmap.org/relation/5761 and so forth.

mboeringa commented 8 years ago

Hi @ThomasEmge,

I am currently attempting to re-render, as I noticed some remaining temp osm files as created by the import tool in my ArcGIS scratch folder.

There may be something else in play here, like a failed import due to running multiple ArcMap sessions...

Anyway, I have finished another big import of the Netherlands I need to have a closer look at. One thing I did notice there, was that in the Vondelpark in Amsterdam, some of the islands, that are inners of waterbodies, did not show up as islands with green grass (as they used to, as I rendered this before without issues), but now showed up in blue watercolor instead. That isn't right. However, overall, the results there were much better than what I saw happening in Dresden, but not as it used to be.

I still have the feeling there may be some issue lurking here...

mboeringa commented 8 years ago

@ThomasEmge,

Could you try and import this multipolygon: http://www.openstreetmap.org/relation/3164798. It is an artificial lake in the Netherlands, and it is completely missing from my rendered data. I do see the outer way without tags in the Polygon feature class, but no multipolygon representing the lake itself. It looks good in JOSM, and no validation errors on the multipolygon object.

Looking at my rendered data of the Netherlands, it is almost like if the number of inner rings of a multipolygon is equal or exceeds the number of processes spawned in the multiprocessing, then that multipolygon isn't properly processed, like if an index variable keeping track of stuff is inappropriately reused. I do see simpler multipolygons with just 1 or 2 inners...

ThomasEmge commented 8 years ago

Here is what my lake looks like when loaded as the only feature in the osm file:

lake

The number of processes should not come into play here as each load iteration builds on the results on the previous step. Nodes are handled first and the merged into one feature class, then the ways are loaded next from the merged point feature class. If there is a node missing from a way, the way is dropped. Then the ways are merged. Next come the relations which are build from the merged lines and/or polygons, and then the relation results are appended into the lines and polygons. After that we are doing it one more time for super-relations. I am currently running larger datasets to see if there is a difference. I'll keep you posted.

mboeringa commented 8 years ago

@ThomasEmge

I can confirm that the particular issue here was caused by an XAPI query time out, and incomplete downloading of an osm XML file, causing missing relations. The image below shows the proper rendering after re-downloading the data.

I will close this one for now, however, I am going to look a bit further at a potential other problem with some data in the Netherlands.

dresden_proper_rendering