Suggestions for converting to format usable in ArcGIS?

[MattWLeonard]

Hi, and thanks for posting all this data! It should be very useful as described, but I'm having trouble opening/converting the JSON file (I'm trying to use the Georgia.json file). I've tried opening it in QGIS, but I think the file is so large that it crashes the program. In ArcMap, I've tried the JSON to Features tool, but I get "unexpected error" - not sure if this is due to file size or what. I also tried the ArcGIS Data Interoperability Quick Import tool, but it doesn't list JSON as an input file type.

It would be great if this data were broken up into smaller pieces, instead of an entire state in one big chunk...but since that's not currently available, any suggestions on how to use this huge file would be appreciated!

[avf-1992]

Hi there, I am having a similar problem. I tried converting the JSON to a geojson file yet both online converters I tried never returned an output. Additionally, I ran the JSON to Features geoprocessing tool to no avail (same error as OP) both within an mxd and in ArcCatalog. I can't bring the JSON into ArcGIS as-is. I downloaded Maine.json

If you have any solutions to downloading, converting, or accessing this data, please help!

Thanks!

[MattWLeonard]

^ FWIW, I've been told by 2 friends (who are more knowledgable of the file structure) that it already appears to be a geojson file, even though the extension is only .json.

[avf-1992]

It does have a geometry type (polygon), which I think is one of the prerequisites for it to be a geojson file. So at least that makes sense! Thanks!

But about the not-being-able-to-access-it....

[bowlesgsrcx]

When using the JSON to Features tool in ArcGIS Pro, I received the following error: 001558: Error parsing .json file . Description The JSON structure does not conform to ESRI-JSON standard for feature sets. Solution Review the JSON structure in the input file. The JSON must have at least the geometryType, spatialReference, fields, and features (with geometry and attributes) property.

Unfortunately, I could not open the json file to edit it as it is too large. --gary

[nmpeterson]

I've had success using this python toolbox from within ArcGIS Pro. (My fork of the project has been updated for Python 3 compatibility, as the original has been archived.) I was unable to use it in ArcGIS Desktop because of memory limitations.

[jharpster]

We don't use Arc that often. Ian Dees split up the files into census tract bite size chunks here: https://twitter.com/iandees/status/1013791681945198594

Also there is a discussion ongoing over at reddit here.

[germrothdaniel]

I've written a python (2.7) script that converts the JSON values to polygon features. I'll get it packaged up and posted tomorrow. I have no idea how to use github so can anyone give me any advice? Do I add it to this project or make my own repo and post the link to that here?

[jharpster]

I suggest you make your own repo and post a link to it here so others can find it. Thanks!

[germrothdaniel]

Done! https://github.com/germrothdaniel/MicrosoftBuildingsToFeatureclass

[kannes]

You can load these files in QGIS just fine. It simply takes "a while" and due to the lack of spatial indexing, even when zoomed in, it is not fun. But you can convert them to a different format like GeoPackage or a Shapefile to get the benefits of those.

If your QGIS runs out of memory, it might be due to an older version of GDAL. Use GDAL 2.3.x for extreme memory savings when dealing with JSON data.

I made a single GeoPackage (25G) you can grab at https://www.datenatlas.de/geodata/public/sources/usbuildingdata.blob.core.windows.net/usbuildings/California.zip and I am also running conversions of each single file to GPKG and Shapefiles now. No guarantees that these will stay up for long, mostly meant as PoC. ;)

[MattWLeonard]

I figured QGIS might just take a while to load the statewide file, so I let it try for 2 hours while I was doing something else on a different machine, and still nothing. But now I got the tract-size geojson files from Ian Dees which were linked above, so I'm good to go!

[avf-1992]

Thanks all! I will check out some of the options and the Reddit thread, hopefully it works out for me! (I have a feeling it will.)

[avf-1992]

@nmpeterson THANK YOU for the python toolbox you recommended. I was able to use it in 10.5, as we don't use Pro yet at work. The only issue I had was trying to run the Import GeoJSON File tool with the existing folder name "geojson-madness-master". It gave me an error "ExecuteError: ERROR 000354: The name contains invalid characters". So, I changed the hyphens to underscores (geojson_madness_master) and re-ran the tool. Worked like a charm and I now have the data accessible as a polygon feature class on my machine!

Thanks again! Y'all are awesome.

[bradysmith66]

Using QGIS worked to export to shape worked for me to get the file into a shapefile format to use in ArcGIS. After comparing to our jurisdiction's footprint layer and photo collection, I am assuming that the photos for our state (Oregon) was from around 2012. Is there documentation showing the date the were derived from?

[ggirton]

I used the R package simple features (sf) to write a shape file and load into Scan/US (proprietary mapping, but it can read Arc shapefiles) to look at, it was just 3 lines:

json_in <- setfilepath("DistrictofColumbia.json", fromdir)
sf.buildings <- read_sf(json_in)
write_sf(sf.buildings,"DistrictofColumbia.shp")

[vvascanus]

For splitting a jile into manageable arbitrary-sized chunks, use GSplit3, http://www.gdgsoft.com/gsplit/ Worked like a charm. Just copy the first three lines from the first part into the subsequent parts,

[CJuice]

Built off of @germrothdaniel 's repository. Made a standalone python script, no ESRI Toolbox tool. Was built using ESRI ArcPro Python 3.6 version. Edit parameters, as indicated by comments in code, and run it from an IDE. Ran it successfully on Maryland dataset of approximately 1.6 million polygons without issue. https://github.com/CJuice/MicrosoftBuildingsToFeatureclass

[slibby]

With the new 1.1 release today, the files now convert successfully with JSON to Features in ArcGIS Pro

[jgravois]

With the new 1.1 release today, the files now convert successfully with JSON to Features in ArcGIS Pro

very cool! my guess is that the initial error was caused by #15