search

Open-Historical-Map-Labs / openhistoricaltiles

First iteration of vector tiles from OHM Planet data
BSD 2-Clause "Simplified" License
3 stars 0 forks source link

Islands were being forced to POINT geometry #59

Open gregallensworth opened 4 years ago

gregallensworth commented 4 years ago

The DB table osm_island_polygon does not in fact contain polygons, but points. I've been asked to figure out why newly-drawn islands aren't appearing, and this is certainly one piece of that.

Relevant to #57 which came later than this, about styling islands.

┆Created By: Dan Rademacher

gregallensworth commented 4 years ago

https://github.com/OpenHistoricalMap/openmaptiles/commit/b9b0786a8cf86b01f2237ed7d8bb7ba3f93cbd96

Confirmed that this was an intentional behavior in OpenMapTiles: update_osm_island_polygon()

I removed this, and later confirmed when diffs were ingested, that islands are now kept as polygons.

gregallensworth commented 4 years ago

Still to do, though: Many islands are in the DB, which already had their geometry flattened to a point. There are too many to simply go into OHM and make simple do-nothing edits to each one so they picked up by diffs... Ideas?

select count(*) from osm_island_polygon where st_geometrytype(geometry) = 'ST_Point';
31974
danrademacher commented 4 years ago

I wonder if there's some reason these were being forced to points in OpenMapTiles. Looking at this, https://github.com/openmaptiles/openmaptiles/commits/master/layers/place/update_island_polygon.sql, almost seems like they were focused on getting points on which to hang labels or something. But dropping the geometry from all islands... can't imagine that was the intention. Strange.

So as you point out to get the geoms here, we'd have to prompt some special case re-ingestion of these. Hmm

danrademacher commented 4 years ago

Seems like this should roll into #64

danrademacher commented 4 years ago

Or maybe not re Carto work in #64, as Gregor just pointed out to me, it’s a database/sync/diff issue in the data and not in the map style. Though there’s some overlap, as islands are a style.

danrademacher commented 4 years ago

So this is more challenging than I realized:

Islands were being forced to points for some time, and 33,000+ islands still are. Could an update be forced so they send diffs, and thus fix drawing islands as vector tiles?

So even if we change how we consume islands, the only way we'd reconsume them is if they are somehow edited and thus become part of a diff.

Gregor:

Hopefully they can do something as simple as an UPDATE query to set the records to last-updated-right-now, and let nature take its course.

Hmm, sounds easy but I wonder about ancillary impacts of an edit that's really only for the purpose of our renderer...

danrademacher commented 4 years ago

The alternative is to purge our entire database and start over. Which is possible, but time consuming and maybe best done after we do some carto implementation and isolate and account for any other issues we might not be consuming.

But the more I think about the idea of a force change upstream, the more I worry we’ll regret it later. Things Jeff might look at down the road, like average edits per month/year etc would have this huge artificial spike when every island in the world was changed

We're guessing a day’s work in generating a world file, purging the DB, and reloading from scratch. Plus a bit of time remembering how we did it the first time...

jeffreyameyer commented 4 years ago

There are actually many reasons for looking at a re-import of the entire planet's set of natural features. The first time I did this was rather messy and probably too rushed. So, please don't dismiss that too quickly... it might be the right move.

Also, you're guessing far too little time, imho. Closer to a week of careful:

Just my guess, but maybe I'm too slow at these things!! High risk for screwing things up, so just want to be careful. : )

On Mon, Sep 30, 2019 at 9:39 AM Dan Rademacher notifications@github.com wrote:

The alternative is to purge our entire database and start over. Which is possible, but time consuming and maybe best done after we do some carto implementation and isolate and account for any other issues we might not be consuming.

But the more I think about the idea of a force change upstream, the more I worry we’ll regret it later. Things Jeff might look at down the road, like average edits per month/year etc would have this huge artificial spike when every island in the world was changed

We're guessing a day’s work in generating a world file, purging the DB, and reloading from scratch. Plus a bit of time remembering how we did it the first time...

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/OpenHistoricalMap/openhistoricaltiles/issues/59?email_source=notifications&email_token=AALM4EWCQQS5QFUCOQ4A5PTQMITUDA5CNFSM4IUN4LMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD76JAIA#issuecomment-536645664, or mute the thread https://github.com/notifications/unsubscribe-auth/AALM4EXSGSKW3KU7ZIIFCGTQMITUDANCNFSM4IUN4LMA .

-- Jeff Meyer 206-676-2347 osm: Open Historical Map (OHM) http://wiki.openstreetmap.org/wiki/Open_Historical_Map / my OSM user page http://www.openstreetmap.org/user/jeffmeyer t: @OpenHistMap

bertdeb commented 4 years ago

I'm fully behind any effort to get the other 4 great lakes imported ;-)

danrademacher commented 4 years ago

Hmm I think we’re talking about different things. GreenInfo is referring to dropping the database that renders vector tiles and reimporting the full planet from OHM. But still whatever is in OHM is what we get.

Sounds to me like you’re talking about actually updating the source OHM data... from OSM? Don’t really want to go there for our map rendering work

danrademacher commented 4 years ago

Also I don’t really understand the history of dealing with licensing in the case of importing from OSM To OHM. But in any case that’s a quite different project from what we’re taking about

jeffreyameyer commented 4 years ago

I think what Bert is saying is that:

I think this is more of an end user (i.e. me or others) task, rather than a GreenInfo task, as it relates to data & imports.

On Tue, Oct 1, 2019 at 11:27 AM Dan Rademacher notifications@github.com wrote:

Also I don’t really understand the history of dealing with licensing in the case of importing from OSM To OHM. But in any case that’s a quite different project from what we’re taking about

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/OpenHistoricalMap/openhistoricaltiles/issues/59?email_source=notifications&email_token=AALM4EWJYIFBNDF7FCOT3CDQMOJCRA5CNFSM4IUN4LMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEACIMQY#issuecomment-537167427, or mute the thread https://github.com/notifications/unsubscribe-auth/AALM4EQWMYAR54T5N6NFZKTQMOJCRANCNFSM4IUN4LMA .

-- Jeff Meyer 206-676-2347 osm: Open Historical Map (OHM) http://wiki.openstreetmap.org/wiki/Open_Historical_Map / my OSM user page http://www.openstreetmap.org/user/jeffmeyer t: @OpenHistMap