osmjs - strange polygons

[fre391]

Hi,

first of all thank you very much for sharing osmjs.

I have two files: bremen.osm ~ 127MB islas_canarias.osm ~ 205MB

Test1: When I start osmjs for bremen.osm a result is created, but the poylgons are partly corrupted: ./osmjs -2 -m -l sparsetable -i js/osm2shape.js -j js/config.js bremen.osm 1st pass finished ... geom build error: multipolygon invalid geom build error: invalid ring ... Done Result as screen (ex. water and waterpoly only): http://freyt.de/Bildschirmfoto.png All corrupted polygons have one vertex at coordinate: ~ 214.748,214.748 I could imagine, that these polygons are invalid, because just cutted as exported before.

Test2: When I test something like this ./osmjs -l sparsetable -j js/roads.js islas_canarias.osm This exits without error. But result is a as screen: http://freyt.de/Bildschirmfoto2.png All corrupted polygons have one vertex at coordinate: ~ 214.748,214.748 In this case I'm pretty sure, that these polygons are NOT invalid, because its an isle. Therefore they are isolated and will not cutted in between?

Test3: ./osmjs -2 -m -l sparsetable -j js/shape_export.js islas_canarias.osm ./osmjs -2 -m -l disk -i js/osm2shape.js -j js/config.js islas_canarias.osm ./osmjs -2 -m -l array -i js/osm2shape.js -j js/config.js islas_canarias.osm 1st pass finished geom build error: no rings terminate called after throwing an instance of 'std::range_error' what(): geometry for nodes not available Abgebrochen (Speicherabzug geschrieben) In all these tests no files are created. Might be something like a memory error? I use Ubuntu 12.04 LTS (64bit) in a VirtualBox with base memory 6411MB.

Well, I have also uploaded those oesm files onto my server: http://freyt.de/bremen.osm http://freyt.de/islas_canarias.osm

Would be nice to get some feedback. I'm pretty new to osmjs.

Many thanks Markus

[joto]

Looks like some node coordinates are missing so they get the invalid coordinate 214.748,214.748. How did you generate those extracts?

[joto]

You might be missing the completeWays option for osmosis.

-l array needs more that 12GB these days so it will not work on your machine.

[fre391]

Hi Jochen, thanks for fast reply...

The bremen.osm was taken from geofabrik. Normally they dont use completeways as far as I know. Therefore It wasn't a suprise for me that in this case also the result of osmjs has some artifacts....But, I have used osmosis for the canarias. Yes, Normally I use "completeways" as Im aware about the problems without. Might be possible, that I missed it indeed, because I assumed thats not necessary for an isle.I will check it again in the next days...Thanks for this hint....

Yes, I thought "array" would be too much for my given memory. Therefore I have tried "disk" as well, but I ran into the same problems.. Anything I could test?

[woodpeck]

I quickly checked the islas-canarias file and found that it does not have referential integrity; I did

osmosis --read-xml islas_canarias.osm --bb left=-100 right=100 top=80 bottom=-80 clipIncompleteEntities=true --write-xml ic2.osm

which removes references to non-existing objects from ways and relations, and there are about 18,000 such references. Examples of relations missing members are 1759448, 1543921, 1506142; examples of ways missing nodes are 5010228, 6168454, 28697746. Try to run the same command as I did, and process the resulting ic2.osm. Then try and find out why data is missing in your islas-canarias file.

[fre391]

Hi,

yep, you were right. Meanwhile I have performed much more tests with correctly cutted OSM files and everything went fine. I have converted a lot of files meanwhile. During testing I found another issue, which might be interesting. I couldn't solve it. Also --no-repair didn't help out. Here is what I have done. Result was converted nice, but some polygons were missing. I checked the polygon id 4275374 with qgis, but it doesn't report an error....

Command: ./osmjs -l sparsetable -2 -m -j js/test.js bremen.osm

Output: Begin 1st pass finished geom build error: multipolygon invalid geom build error: multipolygon invalid geom build error: invalid ring geom build error: multipolygon invalid geom build error: invalid ring geom build error: invalid ring Done

Result: polygon id 4275374 wasn't included in the shapefile Its tagging in OpenStreetmap is: leisure=park ... name=stadtwald */

function counter(zahl, l, f) { var tmp = zahl + '';
if(tmp.length > l) l = tmp.length;
if(typeof f == 'undefined') f = ' '; while(tmp.length < l) tmp = f + tmp; return tmp; }

var test=0; var shp_test = Osmium.Output.Shapefile.open('test','polygon'); shp_test.add_field('id','string',38);

Osmium.Callbacks.area = function(){ if (this.tags.leisure){ /positive tags/ if (this.tags.leisure=='park') { test++; shp_test.add(this.geom, { id: '{'+counter(test,8,0)+'}' }); } } }

Osmium.Callbacks.init = function(){ print('Begin');

}

Osmium.Callbacks.end = function(){ shp_test.close(); print('Done'); }

What do you think? Was the polygon not ok? I recognized also some more polygons missing...

[joto]

There are many reasons why multipolygons can be broken. Especially when you are working on extracts instead of the full planet file. Maybe http://tools.geofabrik.de/osmi/?view=multipolygon can help you. I'll close this issue now as there seems to be no actual bug in Osmium as far as I can tell.

[fre391]

HI Jochen,

In the tool mentioned I can indeed see exactly this polygon to be reported as invalid. Great tool! Thanks for sharing the link.

Sure, we can close this one.... Thank you very much again... Markus