search

Esri / arcgis-osm-editor

ArcGIS Editor for OpenStreetMap is a toolset for GIS users to access and contribute to OpenStreetMap through their Desktop or Server environment.
Apache License 2.0
399 stars 131 forks source link

"Load OSM File" tool no longer creates spatial indexes for the Line and Polygon feature classes #134

Closed mboeringa closed 8 years ago

mboeringa commented 8 years ago

I noticed that with the latest updates, the "Load OSM File" tool no longer creates a spatial index for the Line and Polygon feature classes stored in the feature dataset. The Point feature class does seem to have a spatial index.

This is most likely a regression, as a review of older imports shows proper spatial indexes on all three feature classes.

ThomasEmge commented 8 years ago

What are the limitations of not having a spatial index? There reason they are omitted because the tool was making wrong guesses in the first place and was causing rendering and storage issues.

mboeringa commented 8 years ago

Poor query / display performance in ArcMap and as web (base)map, or poor export-to-PDF performance on large datasets?

I am slightly puzzled by your remark. I fully agree spatial indexing is not guaranteed to give the best overall performance displaying datasets in ArcMap and ArcGIS Pro, it can heavily depend on proper attribute indexing or spatial (de-)fragmentation as well, but then again, ESRI seems to have always recommended having one, and ESRI tools like Analyze Map (a default menu option in the File menu of ArcMap), clearly test for the presence of a spatial index when preparing the dataset for publication on ArcGIS Online or as some kind of web basemap.

Actually, I only found out about this issue because I accidentally clicked the "Analyze Map" menu item instead of the "Export Map" menu option in the File menu (the options are directly above each other). When I suddenly saw 350 "Warnings" pop-up concerning the lack of a spatial index, it became obvious something had changed / was wrong, as I never saw this before.

Anyway, your remark about rendering and storage issues is interesting as well. What precise issues have you seen as caused by spatial indexing on the generated datasets? Does it have to do with some of the very large to giant geometries in OSM causing issues if spatially indexed?

ThomasEmge commented 8 years ago

I discovered last fall that certain geometries were silently dropped from the loading process when the (wrong) index existed. It might actually have been the "spaghetti" geometry you noticed earlier, not 100% sure though.

mboeringa commented 8 years ago

I discovered last fall that certain geometries were silently dropped from the loading process when the (wrong) index existed.

What about adding the index only at the very end of the loading process? Would it still be a cause of issues?

It seems you may be referring to the generation of multipolygon relations, a process that can only take place after the ways have been loaded. Do you mean a spatial index already existed on the intermediate dataset with just ways, and that the invalid spatial index caused the drop-out of multipolygons (as I also reported in previous issues)?

ThomasEmge commented 8 years ago

At the very end might work, I'll check. The index existed previously on lines and polygons and it didn't take much to trip the loading routine. Unless anyone was looking for specific features you wouldn't notice a couple hundred missing features.