Comparison of import height values to SF building footprint, LIDAR.

geochetan commented 8 years ago

I compared the proposed height values to be imported from SF's official building footprint and LIDAR elevation values. Out of 200 buildings we sampled, the proposed height values were very low compared to the official building height.

Process

Extract building polygons/multipolygons from latest osm pbf. Select random building samples.
From the sample buildings, extract the LIDAR height values using zonal statistics to get the min, max, mean, stdde.
Extract from SF's official building footprint the height values by getting the difference between maxheight and minheight.
Visually inspect the data and compare to the proposed height data in JOSM.

building_height
Building height comparison (n=200). SF building footprint (yellow), LIDAR max (green), proposed import (blue)

Observations

Proposed height values were very low compared to the official building and LIDAR maximum values.
LIDAR max values almost perfectly matches the official building values.
Proposed height are generally good for simple buildings (i.e. residential buildings)

sf_lidar

But fails really bad for complex multipolygon and when there are courtyards.

screen shot 2016-09-01 at 12 10 57
_SF building (65), LIDARmax (63), Import (42)

Proposed changes

Instead of using building centroid as reference for height values, let's use the maximum height for all pixels within the polygon. This complies with OSM best practice of using the maximum height for simple 3d buildings.

Proposed workflow

From the building polygon, create an inside buffer of ~2 meters. This is to avoid getting the values of adjacent buildings.
Compute the zonal statistics for each buffer.
Use the maximum value as height value to be imported in OSM.

cc/ @bdon @maning

bdon commented 8 years ago

@chtnha @maning nice analysis! however the "official building footprint data" is itself pretty inaccurate if you compare it to Mapillary or OpenStreetView - the values are way too high in some places for one or two story buildings. The place I noticed discrepancies was around the 9th Avenue / Irving intersection. So I think the analysis needs to be done against real world imagery and not the 2011 synthetic dataset.

bdon commented 8 years ago

I think the proposed change to determining the height is good. Will work on this change and add to github.

The ideal situation is we can have a local mapper in SF ground truth ~3 areas with different types of buildings/tree cover, or we can determine heights from OpenStreetView/Mapillary.

bdon commented 8 years ago

@chtnha @maning Please check out this section i've added on the wiki: https://wiki.openstreetmap.org/wiki/San_Francisco_Building_Height_Import#Is_the_data_accurate.3F

geochetan commented 8 years ago

We ran a comparison between newly proposed LiDAR import and Mapillary photos (building:level) for 86 buildings. These buildings were taken from residential, downtown, and industrial neighbourhoods of SF.

The below graph shows that the proposed import is higher than the building:levels. We are assuming 3 meter for each building:level.

height

The below graph shows that 80% of the buildings have a difference of less than or equal to 4m when compared with proposed import data.

diff

Observations

Multipolygon buildings are not included in the import.

Edge cases:

We are ignoring the underground buildings like below one. This is a underground train station tagged as building.

screen shot 2016-10-26 at 4 15 36 pm

Buildings with greater ceiling height: LiDAR captures the height well, but the 3m height we consider for each building:level doesn't augment well in such cases

screen shot 2016-10-26 at 6 08 24 pm

cc @bdon @maning @ramyaragupathy

bdon commented 8 years ago

@chtnha wow, nice!

1) how did you determine the height from Mapillary? 2) what locations were these? 3) for the cases in which the proposed import height is wildly off, is this identified easily in LIDAR? Or could it be that the building shapes are too complex for the footprints?

Brandon

geochetan commented 8 years ago

@bdon These are my answers to your questions.

1) We determined the height of building by it's levels. (1 level = 3m) 2) These locations were in Sunset District (residential area), downtown (where more skyscrapers are located), cow hollow (residential area), and the industrial area of SF 3) yes! We can identify these buildings through LIDAR but can't determine through Mapillary street coverage.

bdon commented 8 years ago

@chtnha

1) Using 3m as a level might work for tall commercial buildings but I'm not sure it will result in an accurate height determination for 2-3 story residential buildings in san francisco - these often have tall roof parts or half-height building sections. 2) Can you upload the source imagery and locations to the git repo? When you show the graph of all 86 buildings, they're displayed on the X axis - does this mean the buildings are adjacent? (I'm assuming not since they're from 3 different areas) 3) What I mean to ask, is in the cases where you found that the height was wrong, is there some clue a mapper could use by looking at the imagery to identify that the tagged height is wrong?

maning commented 8 years ago

Comparison results posted in the wiki. https://wiki.openstreetmap.org/wiki/San_Francisco_Building_Height_Import#Building_Survey

No next actions. Closing.

osmlab / sf_building_height_import