Alternative to GeoJSON?

[dkm319]

I'm working with the Minnesota data file and having trouble loading it into ArcMap (10.7). The JSON to Features conversion tool does not recognize .geojson files, only .json files. I tried opening the file in a text document and re-saving it as .json and that did not work. I tried bringing it into QGIS and it crashed :(

Any ideas or any way you could include a .json file in the downloaded folders, as well as the .geojson?

Thanks!

[chris]

Since you're using an ArcGIS product, maybe a shapefile will import better? If you have the GDAL command line tools, you could convert the GeoJSON file to a Shapefile with something like:

ogr2ogr -f "ESRI Shapefile" Minnesota.shp Minnesota.geojson

In general, most of these are quite large files, so opening the entire state in QGIS or similar may not be very workable. There are a variety of different ways you can do this, but I'd boil this down to two basic categories:

1. Load the data into PostGIS and then do queries on it for what you're after. You can also query for the geometries within an area you're studying and then export that back out to GeoJSON.
2. Write a program that reads in the GeoJSON and filters it similarly to PostGIS mentions above.

I've also seen some GeoJSON "splitting" utilities out there. I haven't used any, so don't know much about them, but they may be a way to simply break the single state file up into say 10 smaller files.

[andwoi]

I suspect QGIS is crashing because the file is too large. As @chris mentioned, you can split the geojson files into smaller chunks and work with it that way. Here's a python snippet that will look for individual polygon features and write line-delimited geojson (geojsonl) that QGIS can work with. This doesn't require additional modules over python's standard library. Hope that helps!

import json

# decompressed geojson file. 
with open("Minnesota.geojson") as inf:
    counter = 0
    batch = 0
    # adjust this value to change the number of features per file.
    buildings_per_file = 500_000

    file_template = "Minnesota-{}.geojsonl"
    target_file = open(file_template.format(batch), 'w') 
    line = inf.readline()
    while line:
        # filter to look for polygon features
        if 'Polygon' in line:
            line = line.strip().strip(",")
            parsed_line = json.loads(line)
            geom = parsed_line['geometry']
            target_file.write(f"{json.dumps(geom)}\n")
            counter += 1
        if (counter > 0 ) and ((counter % buildings_per_file) == 0):
            target_file.close()
            print(f"completing batch {batch:,}. total lines extracted: {counter:,}")
            batch += 1
            target_file = open(file_template.format(batch), 'w') 
        line = inf.readline()
    target_file.close()
    print(f"completing batch {batch:,}. total lines converted: {counter:,}")

[alasdairrae]

If anyone's looking for files that will load into QGIS (or similar software) quickly then I have a set I batch converted to GeoPackage (gpkg) with gdal - here they are: https://automaticknowledge.co.uk/us-building-footprints/

(note that they are not the very latest iteration of the files)

This is what I used to batch convert the files didn't take too long do do the whole set, maybe a few hours, and now I can load single states into QGIS much more quickly

for %f in (*.geojson) do ogr2ogr -f "GPKG" %~nf.gpkg %f

[alasdairrae]

For anyone looking for files that load into QGIS (or other GIS software) much faster than the original geojson files, then we have produced an updated set of geopackage files for each state, available here:

https://automaticknowledge.co.uk/resources/#USBuildings

These were converted to gpkg from geojson and we also added in county codes and names so that users can filter by county if they need to.

[npr99]

Thank you!