search

OpenHistoricalMap / issues

File your issues here, regardless of repo until we get all our repos squared away; we don't want to miss anything.
Creative Commons Zero v1.0 Universal
18 stars 1 forks source link

Atlas of Historical County Boundaries (US counties, Newberry Library) #418

Open jeffreyameyer opened 2 years ago

jeffreyameyer commented 2 years ago

Dataset Info Name: Atlas of Historical County Boundaries Source URL: https://digital.newberry.org/ahcb/ License: public domain Geo area: United States Timeframe: 1787-present

Why will this help OHM's data quality - comprehensiveness, accuracy, better licensing/copyright status? This will add historical county boundary information for the history of the United States to OHM. The historical state boundaries are already imported, but those will need to be updated with the county-friendly ways (i.e. ways that are broken by county boundaries).

Import Considerations What existing OHM data might this conflict with? Will it impact any existing relations or create unnecessary duplication? The plan below must account for avoiding potential conflicts. This import will need to respect existing state-level relations. The current plan is to retain existing state-level relations and then update the member ways underneath.

Import-specific tags Will there be any tags specific to the import of this data? Please include potential mapping of source metadata to OHM tags, as well as OHM-specific tags.

Import tags: Applied to relations and ways. key=value license=CC0 / public domain source:name=Newberry Library Atlas of Historical County Boundaries source:license=CC0 / public domain

Relation tag mapping: Source OHM
name name
abbr_name name:abbr
full_name name:official
id_num source:id
'id' source:id_text
citation source:citation
version source:version
cause start_date:cause
start_date start_date
start_date:cause start_date:cause
end_date end_date
start_n preceded_by
terr_type place
cnty_type place
- wikidata
- admin_level
- type=boundary
cross_ref -
fips fips

Way tag mapping: No special way tags outside of import-wide source information.

Import Plans Provide the planned steps for collecting, processing, tagging, linking, and uploading the data Please note: linking data is critical for making OHM a valuable resource across the web. Every step should be taken to link imported data before actually importing it. Be sure to include how the OHM community will be notified of the planned import

1ec5 commented 2 years ago

@jeffreyameyer retagged the state boundaries to license=CC0 / public domain in changeset 39,638. This is just the state boundaries, not the county boundaries that this issue proposes to import, so just an FYI.

markconnellypro commented 2 years ago

Sorry for dropping off the planet for a bit, but I've uploaded all the files that were in the Google Drive ZIP files (except the log file) to Github:

https://github.com/markconnellypro/ahcbp-topology

They are also archived in Zenodo, and, can be cited via this DOI:

https://doi.org/10.5281/zenodo.7080770

Is it ok if I go ahead and delete the Google Drive versions?

jeffreyameyer commented 1 year ago

@markconnellypro - apologies for taking so long to follow up here - I've grabbed those files & having them in github is great. Please feel free to delete the Google zip files & thanks again for your help!

jeffreyameyer commented 1 year ago

From @1ec5's slack message:

I don’t have a ton of experience doing this, but the shapefile has enough information to create the ways in OSM. If each way in the OSM XML file is given a negative ID corresponding to the EDGE_ID, then we could generate relation XML elements from output_counties_ways.csv and output_counties_metadata.csv and append them to the OSM XML file – just text processing in a script using csvkit or similar. The members in those relation elements would pair the negative IDs and the inner/outer roles from the latter CSV. Once the OSM XML file is put together, open it in JOSM and run the validator to ensure that the relation roles make sense. Does this sound feasible? I think there’s still some work to massage the columns in the metadata CSV to meet OHM standards; I think you’d have the best context on that.

1ec5 commented 1 year ago

In case the shapefile doesn’t structure the boundaries as edges, https://github.com/codeforsanjose/OSM-SouthBay/issues/28#issuecomment-1256873120 details a workflow for turning a shapefile of adjacent polygons into a typical OSM structure in which many boundary relations can share a way.

markconnellypro commented 1 year ago

Just wanted to mention that there is a new release of the markconnellypro/ahcbp-topology repository. It now contains 2,000 fewer ways, but the IDs have all changed (this was partially intentional as I was having somewhat surprisingly different results on PostgreSQL 15 versus 14 and decided to reference the FIPS in the IDs).

jeffreyameyer commented 1 year ago

Mark - thank your for keeping us posted on this. I've been dragging on getting this updated, so we'll be sure to get it in using this update.

Muchas gracias!

On Mon, Sep 11, 2023 at 10:22 PM Mark A. Connelly @.***> wrote:

Just wanted to mention that there is a new release of the markconnellypro/ahcbp-topology https://github.com/markconnellypro/ahcbp-topology repository. It now contains 2,000 fewer ways, but the IDs have all changed (this was partially intentional as I was having somewhat surprisingly different results on PostgreSQL 15 versus 14 and decided to reference the FIPS in the IDs).

— Reply to this email directly, view it on GitHub https://github.com/OpenHistoricalMap/issues/issues/418#issuecomment-1714990384, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALM4ESCJAL7CPS4YRXFKS3XZ7WQHANCNFSM546TSBWQ . You are receiving this because you were mentioned.Message ID: @.***>

-- Jeff Meyer 206-676-2347 osm: Open Historical Map (OHM) http://wiki.openstreetmap.org/wiki/Open_Historical_Map / my OSM user page http://www.openstreetmap.org/user/jeffmeyer t: @OpenHistMap