pelias / openstreetmap

Import pipeline for OSM in to Pelias
MIT License
112 stars 72 forks source link

"Invalid regex" error about source=https://... tag #571

Closed langbein-daniel closed 1 year ago

langbein-daniel commented 1 year ago

Hi there,

during import, this error came up:

error: [openstreetmap] tag_mapper error
error: [openstreetmap] PeliasModelError: invalid regex test, 46https://www.regensburg.de/leben/wohnen-u-bauen/wohnbauoffensive/karte-wohnbauoffensive/nordgaustrasse-40-52 should not match /https?:\/\//
    at Object.nomatch (/code/pelias/openstreetmap/node_modules/pelias-model/util/valid.js:117:13)
    at Document.setAddress (/code/pelias/openstreetmap/node_modules/pelias-model/Document.js:411:18)
    at /code/pelias/openstreetmap/stream/tag_mapper.js:65:17
    at /code/pelias/openstreetmap/node_modules/lodash/lodash.js:4967:15
    at baseForOwn (/code/pelias/openstreetmap/node_modules/lodash/lodash.js:3032:24)
    at /code/pelias/openstreetmap/node_modules/lodash/lodash.js:4936:18
    at Function.forEach (/code/pelias/openstreetmap/node_modules/lodash/lodash.js:9410:14)
    at DestroyableTransform._transform (/code/pelias/openstreetmap/stream/tag_mapper.js:34:9)
    at DestroyableTransform.Transform._read (/code/pelias/openstreetmap/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:177:10)
    at DestroyableTransform.Transform._write (/code/pelias/openstreetmap/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:164:83)
error: [openstreetmap] {
  "name": {},
  "parent": {},
  "address_parts": {},
  "center_point": {
    "lon": 12.102892,
    "lat": 49.038213
  },
  "category": [],
  "addendum": {},
  "source": "openstreetmap",
  "layer": "venue",
  "source_id": "node/9855644325"
}

To my understanding, this is caused by the source=https://www.regensburg.de/leben/wohnen-u-bauen/wohnbauoffensive/karte-wohnbauoffensive/nordgaustrasse-40-52 tag from one node of the OSM way https://www.openstreetmap.org/way/959734893#map=19/49.03820/12.10364

If I'm not wrong, source=<URL> is a valid tag (https://wiki.openstreetmap.org/wiki/Key:source).

So I think that you are a bit to strict when validating the OSM input and should allow https:// inside the source tag.

missinglink commented 1 year ago

The output you posted refers to "node/9855644325", where did you see way/959734893?

It's picking up an addr:housenumber tag error which seems to now be fixed.

At some point I remember getting frustrated with these errors and so added this code to catch them rather than have Pelias return place names containing URLs 🤦

Screenshot 2023-02-28 at 22 48 41
langbein-daniel commented 1 year ago

Thanks for the quick reply and looking into this! I have overlooked the old Version #1 when viewing that node on openstreetmap.org and was confused as I did not see any https in the current version ...

While wondering where the error was coming from, I then came across a "Way" with the source tag. But that has nothing to do with the error. I didn't consider that my OSM download could be outdated ... Sorry for the false positive.