The NYC Buildings Maintenance Toilin'

Code and helpers for maintaining New York City building footprints in a versioned ESRI Enterprise Geodatabase. Friends, this is our NYC buildings footprints in a versioned ESRI Enterprise Geodatabase toil, our rules, the trick is never to be afraid.

Dependencies
Import
Quality Assurance
Export To GeoJSON
Execute Nightly Maintenance Tasks
Suggested Setup
Attribute Metadata

Dependencies

ESRI ArcGIS Pro python 3.x
geodatabase-toiler on PYTHONPATH

Import

> set SDEFILE=X:\GIS\Internal\Connections\oracle19c\dev\GIS-ditGSdv1\bldg.sde
> set PYTHONPATH=X:\geodatabase-toiler\src\py;X:\geodatabase-buildings
> c:\Progra~1\ArcGIS\Pro\bin\Python\scripts\propy.bat import.py BUILDING X:\conns\bldg@geocdev.sde\BLDG.BUILDING

Quality Assurance

> set SDEFILE=X:\GIS\Internal\Connections\oracle19c\dev\GIS-ditGSdv1\bldg.sde
> set TARGETLOGDIR=X:\gis\geodatabase-scripts\logs\building_maintain
> set PYTHONPATH=X:\geodatabase-toiler\src\py;X:\geodatabase-buildings
> c:\Progra~1\ArcGIS\Pro\bin\Python\scripts\propy.bat qa.py BUILDING

Export To GeoJSON

Additional Dependency:

ogr2ogr on PATH

Consider repeatedly dumping this file to a drive that supports previous versions or to somewhere cloudy.

> set SDEFILE=X:\GIS\Internal\Connections\oracle19c\dev\GIS-ditGSdv1\bldg.sde
> set PYTHONPATH=X:\geodatabase-toiler\src\py;X:\geodatabase-buildings
> c:\Progra~1\ArcGIS\Pro\bin\Python\scripts\propy.bat export.py BUILDING

Execute Nightly Maintenance Tasks

Update any data that editors don't maintain manually
Reconcile and post BUILDING_DOITT_EDIT version to DEFAULT
Compress and rebuild geodatabase administrator indexes
Rebuild buildings feature class indexes and update database optimizer statistics
Run QA on buildings feature class DEFAULT version
Notify the squad of QA results

Update the environmental variables at the top of this batch file as needed.

> sample_maintain.bat

Suggested Setup

Copy the sample batch files out of geodatabase-buildings\geodatabase-scripts into a standalone geodatabase-scripts. Then update variables at the beginning of the batch files to match the environment where we are running.

You were expecting containers? Please.

\gis
   \connections
   \geodatabase-buildings
   \geodatabase-scripts
                       \logs
                            \building_maintain.log
                            \qa-BUILDING-yyyymmdd-hhmmss.log
   \geodatabase-toiler

Attribute Metadata

We maintain these attributes in the geodatabase. Some are published, often under different names and with varying degrees of domain decoding, to NYC Open Data and to the NYCMaps ArcGIS Online organization. Here we will document what may be relevant when performing quality assurance or when attempting to decode published column names.

Attribute	Maintenance Notes and Clues	NYC Open Data	NYCMaps ArcGIS Online
OBJECTID	Synthetic primary key generated by ESRI Geodatabase.	NA	NA
NAME	Unmaintained	name	Name
BIN	Building Identification Number, pronounced "bin" by all Intended to be unique but in daily maintenance is not	bin	BIN
BASE_BBL	Concatenated borough, block, and lot number for the building's tax lot Multiple buildings can exist on one tax lot Some are "pseudo bbls," assigned by the Dept. of City Planning for territory not covered by official tax lots These have specific values like lot number 9999	base_bbl	Base BBL
CONSTRUCTION_YEAR	We do not allow year 0 in the geodatabase NULL is allowed	cnstrct_yr	Construction Year
GEOM_SOURCE	Day to day heads-up digitizing, the default, is "Other(manual)"	geom_source	Geometry Source
LAST_STATUS_TYPE	Set by editors to indicate splits, merges, etc	lststattype	NA
DOITT_ID	Synthetic business key trigger-generated from a sequence See compiletriggers.sql under the sql directory New doitt_ids are generated by new records, splits, splits of splits, multisplits, and pastes	doitt_id	DOITT ID
HEIGHT_ROOF	Current editor protocol is to round to the nearest foot	heightroof	Height Roof
FEATURE_CODE	These are domains, as of May 2023 we publish the codes not the values 1000 = Parking 1001 = Gas Station Canopy 1002 = Storage Tank 1003 = Placeholder 1004 = Auxiliary Structure 1005 = Temporary Structure 1006 = Cantilevered Building 2100 = Building 2110 = Skybridge 5100 = Building Under Construction 5110 = Garage	feat_code	Feature Code
STATUS	This was used in some sort of bulk update but is not maintained	NA	NA
GROUND_ELEVATION	In daily editing this is often input as the mean of the elevations on either side	groundelev	Ground Elevation
CREATED_USER	Generated by the geodatabase	NA	NA
CREATED_DATE	Generated by the geodatabase	NA	NA
LAST_EDITED_USER	Generated by the geodatabase Good for QA flagging who is best situated to review an issue	NA	NA
LAST_EDITED_DATE	Generated by the geodatabase	lstmoddate	NA
ADDRESSABLE	Y, N, or NULL, the value is precisely indicated by the feature_code.	NA	NA
MAPPLUTO_BBL	Joins buildings to Boro-Block-Lots in Dept. of City Planning's MapPLUTO For condos we attempt to use BBLs from the most recently available MapPLUTO release	mpluto_bbl	Map Pluto BBL
CONDO_FLAGS	This useless column is not maintained or published	NA	NA
ALTERATION_YEAR	Set when the last_status_type is "alteration" but then sticks around	NA	NA
SHAPE	Geometry in http://epsg.io/2263 OGC-valid at .0005 US foot tolerance No multipart polygons are allowed No self-overlapping polygons are allowed Buildings may overlap with other buildings because of cantilevers and overpasses	shape	shape
GLOBALID	Auto-generated UUID	globalid	NA

mattyschell / geodatabase-buildings

readme