Define update strategy

[almccon]

This isn't mentioned in the http://wiki.openstreetmap.org/wiki/Import/Guidelines or http://wiki.openstreetmap.org/wiki/Import/Plan_Outline, but I've noticed in recent discussions on the imports list, proposed imports have been asked about their plans for how to deal with updated data in the future.

In our case, we know that we are proposing to import fairly old data (from 2008), and we know that 2014 data exists, but is not available for free (currently).

Is it sufficient to tag the import changesets with the source and the date, or should we also put the date on the features themselves?

Should we preserve the building ID BLD_ID or the AIN attributes, in case they are useful in tracking changes between the 2008 and 2014 data?

Once we've come up with good answers to these questions, we should write them up in our import proposal: https://wiki.openstreetmap.org/wiki/Los_angeles,_California/Buildings_Import

[talllguy]

I'm interested in this as well. In traditional GIS or database design, we'd use a unique ID to manage changes overtime. In OSM, since those ID's cannot be locked it seems like some people don't think it is best to add them. However I see that in the NYC import, they did add a unique ID. I have also seen a tool somewhere on GitHub to look for changes within an area but haven't tested it personally.

Perhaps you can query out the changeset IDs and then filter out anything other than version 1 when its time to update. Anything version 1 would not have been altered by another editor. Replace/update the untouched features as needed, and then anything with individual updates could be manually reviewed against the newer data.

[cityhubla]

Just an update, our strategy is as follows.

1. Issue #3 : Determine which fields will be imported. You can help in identifying OSM tag equivalents and there are also UseTypes values in the assessor's data that can be filtered to OSM keys (Ex: building:levels, values like One Story, Two Story, etc are mixed with other values like Fast Food. Upon agreed list of attributes, proceed to update convert.py
2. Issue #9 : The python script convert.py where address points are added within the building needs work as some points are still place outside of outline.
3. Issue #10 : Develop instructions for importers. The efforts and coding here in this repo will create osm files for importers. Files will be stored via an S3 server and accessed through the OSM Tasking Manager. Depending on the outcomes with the merging process, the instructions would include manual error checks for importers.

let me know if I'm missing any items, and I can update the readme.

[planemad]

Was briefly poking through past imports and looks like no one has ever figured out an update strategy with OSM imports. This is a pretty complex idea to branch and merge geographic data with a crowdsourced database and figuring this out would possible open a flood of data collaboration with OSM.

@almccon have you come across any prior ideas on this being discussed elsewhere or are we thinking this out from scratch?

[planemad]

As a simple rule, one should only auto update data if the datasource is the same. If a building geometry in OSM came from the county snd some attributes were later modified by a local mapper, during an update, only the geometry and attributes that came from the the county should be updated, leaving everything else untouched.

This might require each attribute in OSM having a source/reference information just like Wikidata, but this is a currently a missing piece in the OSM data model.

[stevevance]

@planemad That's kind of what happened in Chicago. When @iandees imported City of Chicago-provided buildings data, his import script didn't overwrite buildings added by any local mapper.

The other thing is that no one has imported buildings en masse in Chicago since then. I wouldn't want that to happen. I like the challenge of building a community here of local mappers who will edit buildings.

The downside is that we must wait for Bing to update satellite imagery to show us where new buildings exist. Well, that's not the only way. There's obviously the field visit, and Mapillary. I've also been using building permits as a tip to tell me where to look for new satellite imagery.

[jschleuss]

@stevevance good points. L.A. County will release a new shapefile of buildings later this fall as soon as the contract with their vendor breaks. At that point we can run a query on only new non-L.A. City buildings. Then set up new tasks for importing/merging those.

But separately we need to build a larger OSM community in Southern California. Hopefully it'll kick off with this import!

[cityhubla]

@jschleuss @stevevance, I'm wondering, with LA City's 2014 dated import, it still leaves out buildings that have been built or demoed in the 2015 year. We can still identify these with the now yearly updates to the LA County Assessor released on the open data portal. Future updates that would continue local community contributions, would have them do a check with OSM and Assessor data to identify the potential changes and can be tasked for mappers as a TM or MapRoulette challange and verified with hopefully bing imagery. Doing a quick check there are 248 potential new buildings in 2015

The good thing about LA County's parcel data is that it has coordinates on the centroid. Does Chicago maintain such a dataset?

[stevevance]

Chicago doesn't update its buildings shapefile annually. The Cook County assessor's data isn't open. My strategy to draw new buildings is to use Bing imagery (which could be showing a building built 0-3 years ago!) for the shape, and buildings permits data for the address.

[jschleuss]

We'll need to update with Assessor 2016 data when/where appropriate.

[maning]

Closing and moving discussion to: https://github.com/osmlab/labuildings/issues/112