Initial commit for missing localities

whosonfirst-data / whosonfirst-data-admin-au

Whos On First admin data for AU, homepage: https://whosonfirst.org

Other

3 stars 5 forks source link

Initial commit for missing localities #10

Closed NickStallman closed 4 years ago

NickStallman commented 5 years ago

This commit was created by:

Downloading the latest PSMA localities data
Downloading the latest WOF data from git
Going over each locality in the PSMA data, and seeing if any WOF localities intersect with it. If there are no intersections then a new WOF locality is generated to fill the hole.

Centroids are generated by MapShaper. Everything is run through wof-exportify before committing. The source if the data is aus-psma as per the sources git repository. It is missing hierachies.

stepps00 commented 5 years ago

Thanks @NickStallman - this is great!

I'd like to take a bit of time to pull each placetype in Australia to verify the changes, which may take me into next week. I'll give this PR priority if things can be merged easily, and since it looks like each record in exportified/formatted correctly, it should be.

I'll circle back when I can give this some time to review.

stepps00 commented 5 years ago

And re hierarchies: I think that if the changes are valid, we can merge and PIP the new records to update hierarchies once merged.

NickStallman commented 5 years ago

Great! I'll start looking at updating polygons to ensure they are up to date for everything I can conclusively match up, and also start looking at conflict resolution where data disagrees.

NickStallman commented 4 years ago

My todo list to tweak in the code:

Set mz:is_current = 1 for all updated records from PSMA data.
Set mps:latitude and mps:longitude mirroring the lbl values
Erase any and src:lbl:centroid replace it with src:lbl_centroid
Set name:eng_x_preferred to the official PSMA name

I don't currently have code for generating hierarchies however this wouldn't be terribly difficult to add. Probably makes more sense for me to construct them and include it in the PR.

thisisaaronland commented 4 years ago

@NickStallman FYI:

As I write this I see that the latter -spatial package hasn't been updated to assume Python 3 yet. I will see whether I can get that done this morning.

thisisaaronland commented 4 years ago

FYI - py-mapzen-whosonfirst-spatial has been updated to support Python 3.

stepps00 commented 4 years ago

That all looks good, @NickStallman

To expand a bit on this point:

Set name:eng_x_preferred to the official PSMA name

If there is ever an existing value in the name:eng_x_preferred property that doesn't match the PSMA name, we should:

Move the existing name:eng_x_preferred name value to the name:eng_x_variant property (either add the variant prop, or append to the list if existing)
Set name:eng_x_preferred to the official PSMA name

That way, we're able to store variant names, rather than simply deleting/removing.

NickStallman commented 4 years ago

Yep sounds good.

There is a corner case I do need to look out for, some Australian suburbs have capitalisation in their name but the PSMA data is all uppercase. E.g. McMahons Point. I'll likely to the comparison case-insensitive to try and preserve this where WOF has the correct name.

stepps00 commented 4 years ago

Replacing with https://github.com/whosonfirst-data/whosonfirst-data-admin-au/pull/24.