pelias / openstreetmap

Import pipeline for OSM in to Pelias
MIT License
112 stars 72 forks source link

OSM records with 'constructor' tag result in invalid Pelias documents #557

Closed orangejulius closed 3 years ago

orangejulius commented 3 years ago

Back in https://github.com/pelias/pelias/issues/867 we noticed that our name.* dynamic mapping had a strange entry:

In addition to expected fields like name.default, name.en, etc there was a dynamic name field with the key function Object() { [native code] }.

I was looking into language related data today and spent an extra moment to track down what was going on here.

It appears that OSM records with the non-standard constructor tag are the ultimate cause of this odd data.

The good news is there are only 60 records in all of OSM with this tag, and only 32 records ended up with the function Object() { [native code] } property in the planet build I was examining.

Still, we probably want to skip the constructor tag when importing :)

orangejulius commented 3 years ago

Here's all the OSM records that currently cause this issue in our builds:

'https://openstreetmap.org/way/361486850', 'https://openstreetmap.org/way/173145217', 'https://openstreetmap.org/way/43664624', 'https://openstreetmap.org/way/317549565', 'https://openstreetmap.org/way/764344502', 'https://openstreetmap.org/way/770844609', 'https://openstreetmap.org/way/736843631', 'https://openstreetmap.org/way/546394548', 'https://openstreetmap.org/way/666638716', 'https://openstreetmap.org/way/87685903', 'https://openstreetmap.org/way/548264277', 'https://openstreetmap.org/relation/11260075', 'https://openstreetmap.org/way/546388878', 'https://openstreetmap.org/way/804562392', 'https://openstreetmap.org/way/731839615', 'https://openstreetmap.org/way/360732599', 'https://openstreetmap.org/way/299651908', 'https://openstreetmap.org/way/764333216', 'https://openstreetmap.org/node/281226115', 'https://openstreetmap.org/way/770844629', 'https://openstreetmap.org/way/831261068', 'https://openstreetmap.org/way/44063283', 'https://openstreetmap.org/way/597564764', 'https://openstreetmap.org/way/736843642', 'https://openstreetmap.org/way/548280145', 'https://openstreetmap.org/way/548262350', 'https://openstreetmap.org/way/315890958', 'https://openstreetmap.org/way/547581846', 'https://openstreetmap.org/way/817810785', 'https://openstreetmap.org/way/548268061', 'https://openstreetmap.org/way/547579822', 'https://openstreetmap.org/way/770844597'

missinglink commented 3 years ago

Funny, nice catch