Ukraine is filled as water

[strump]

Humanitarian map style has some strange bug with Ukraine territory. For some reason it's filled with water color:

Could you please fix it?

[almccon]

Possibly related: Spain and Portugal are filled with some other color. I'm not sure what that color signifies:

[skorasaurus]

@almccon - that is quite odd, that is color used for polygons tagged landuse=industrial . I'm looking into it but haven't found the culprit yet. Since there hasn't been a code change in with anything in landuse recently, I'm guessing it's something with a change in the relations for those two countries and how our stylesheet interprets it. I'll try to track this down when I have more time this weekend.

(I haven't looked at all into the Ukraine water, don't know if it's related, so I'll wait to create a separate issue)

[skorasaurus]

Narrowed down that the only relation that the Spain, Portugal, Andorra share is the Iberian Peninsula relation - http://www.openstreetmap.org/relation/3870917#map=5/38.548/0.813&layers=H I'm having trouble obtaining the relation's history but my guess is that the area=yes tag is the culprit.

[skorasaurus]

The relation containing the area of the water is http://www.openstreetmap.org/relation/3791141

My guess is that one of its members is tagged incorrectly; if you could help track that which one it is, that would be great.

[strump]

@skorasaurus , I've tried to investigate relation 3791141 but can't tell what member cause an error. I've exported all tags and values of this relation members using JOSM: ukraine_3791141_tags.txt And exported all tags of Romania boundary #90689: Romania_90689_tags.txt I've tried to compare this tags and found that Ukraine border has 23 lines "waterway=stream". While Romania border has only "waterway=canal" and "waterway=river". I don't think this could be a problem.

But JOSM validation said that Ukraine border relation is not closed. It has gaps. I've found 6 gaps. Next paired lines are both in relation #3791141 and must be fixed. Gap 1) https://www.openstreetmap.org/way/299003249 https://www.openstreetmap.org/way/124525099

Gap 2) https://www.openstreetmap.org/way/299003229 https://www.openstreetmap.org/way/290354632

Gap 3) https://www.openstreetmap.org/way/299003222 https://www.openstreetmap.org/way/129102890

Gap 4) https://www.openstreetmap.org/way/296298450 https://www.openstreetmap.org/way/170310996

Gap 5) https://www.openstreetmap.org/way/296298479 https://www.openstreetmap.org/way/290366533

Gap 6) https://www.openstreetmap.org/way/296298514 https://www.openstreetmap.org/way/290366513

Could these gaps be a problem for HOT style?

[skorasaurus]

@strump awesome work, thank you for tracking that down! Those gaps would likely be problematic. I see that some of the relation's members had edits reverted just a few hours ago. At higher zoom levels, the water isn't appearing so this appears to be fixed. It will take several hours or even a couple days for the tiles at lower zoom levels to be regenerated.

[pavlo-dudka]

This issue may be caused by special processing of multipolygons that don't have tags by osm2pgsql. See https://lists.openstreetmap.org/pipermail/dev/2014-June/027910.html Relation 3791141 doesn't have any tags except type and useless note. That's why osm2pgsql rely on members tags. It decided to use tags of http://www.openstreetmap.org/way/316760295 (or another having name=Brațul Chilia and waterway=river) because of some reason(first member or least id?). You can see label "Brațul Chilia" along all members of 3791141. I think 3791141 should be simply removed from OSM database since it doesn't describe anything.

[strump]

Relation http://www.openstreetmap.org/relation/3791141 has a note: Ukraine borders for navigators upload. So I think it's used to export Ukraine data for further conversion to the GPS Navigators. I'll ask on Ukrainian forum who is using this relation.

[pavlo-dudka]

Relation can be fixed in some way or even removed, but similar errors will appear again and again. Humanitarian render should be reconfigured somehow. See related issue https://github.com/openstreetmap/osm2pgsql/issues/80 It seems that proper configuration of "Lua tag transformations" will help to do this. See description of "function filter_tags_relation_member" in https://github.com/openstreetmap/osm2pgsql/blob/master/docs/lua.md See sample implementation of "function filter_tags_relation_member" in https://github.com/openstreetmap/osm2pgsql/blob/master/style.lua

[yohanboniface]

Sorry for not being active on this guys, I'm on a field mission in Jordan with not much Internet and time. Thanks for tracking down the issues, and yeah @pavlo-dudka you're right we need to make sure this will no break in the future again, so it is likely that a change in the rendering code itself is needed. Thought I don't think Lua is the option, as we want to use the default osm2pgsql style without custom configuration, to make the style easier to set up and factorise the database with other rendering. We should maybe try to look how it's done on the osm-carto style and why it doesn't break there. I'm back on track next week, and will look at this more in details if still needed at this time.

[pavlo-dudka]

@yohanboniface, please check if you are using up-to-date version of osm2pqsql. It seems that the error was fixed by https://github.com/openstreetmap/osm2pgsql/commit/29fa1d0b438329256ca018fa8f3c492b234c3190

[yohanboniface]

cc @cquest @jocelynj about the osm2pgsql version we are using.

[cquest]

I'm almost sure the one we're using is a bit old and buggy. We're using lua filtering.

[skorasaurus]

@cquest - the HOT osm2pgsql database does not use lua filtering. Do you think we could upgrade the version of osm2pgsql used to generate the database?

[skorasaurus]

Was browsing some past issues and I see that the water no longer appears in Ukraine .

The relation causing this bug was deleted a couple months ago.

To prevent it from happening it again; it may be solved by upgrading osm2pgsql although I wouldn't reflexively do it, things could break without testing first although I admit I don't have time at the moment to find out if osm2pgsql has changed significantly (a quick scan through recent tagged builds just notes that it's).

I agree with @yohanboniface ; I don't see any specific problems that implementing lua would otherwise fix although I'm haven't stayed up to date with osm2pgsql developments nor do I know why the default stylesheet moved to use lua.

[pnorman]

nor do I know why the default stylesheet moved to use lua.

If you mean OpenStreetMap Carto, it hasn't moved to Lua.

Lua tag processing allows you to choose to process multipolygons and other relations how you want.

The relation that was involved in this report has been deleted, and the user firmly told not to recreate it.

I believe the osm2pgsql version on the French servers is before 0.84.0, and has known bugs with multipolygon handling, as well as other known bugs. If someone comes with a bug report involving multipolygon processing, I'd want to know that it could be reproduced on a recent version before trying to diagnose it.

Migrations needed when moving from older versions are listed in migrations.md, but given how long it has been since the initial import on the French servers, I'd recommend reimporting to fix clustering, table bloat, index bloat, and all the geometries that have been broken over time.

[strump]

Looks like this issue is not relevant anymore.