jgoodall / us-maps

GeoJSON and TopoJSON map files
346 stars 104 forks source link

Where to get updated geojson data #8

Open maddadder opened 2 years ago

maddadder commented 2 years ago

I found a link to the updated geojson data: https://www2.census.gov/geo/tiger/TIGER2020/ZCTA5/. If you go up a level you can find even more recent than 2020. Can somebody update this repo with more up to date zips?

jgoodall commented 2 years ago

I attempted to update all the files with 2020 census data.

To get the updated files:

mkdir -p raw/state && cd raw/state && curl -O https://www2.census.gov/geo/tiger/TIGER2020/STATE/tl_2020_us_state.zip && unzip tl_2020_us_state.zip && cd ../..
mkdir -p raw/county && cd raw/county && curl -O https://www2.census.gov/geo/tiger/TIGER2020/COUNTY/tl_2020_us_county.zip && unzip tl_2020_us_county.zip && cd ../..
mkdir -p raw/zcta5 && cd raw/zcta5 && curl -O https://www2.census.gov/geo/tiger/TIGER2020/ZCTA5/tl_2020_us_zcta510.zip && unzip tl_2020_us_zcta510.zip && cd ../..
mkdir -p raw/hrr && cd raw/hrr && curl -O https://data.dartmouthatlas.org/downloads/geography/HRR_Bdry__AK_HI_unmodified.zip && unzip HRR_Bdry__AK_HI_unmodified.zip && cd ../..
mkdir -p raw/hsa && cd raw/hsa && curl -O https://data.dartmouthatlas.org/downloads/geography/HSA_Bdry__AK_HI_unmodified.zip && unzip HSA_Bdry__AK_HI_unmodified.zip && cd ../..

Then to convert to GeoJSON:

rm -rf geojson && mkdir -p geojson && cd geojson
ogr2ogr -f "GeoJSON" state.geo.json ../raw/state/tl_2020_us_state.shp tl_2020_us_state
ogr2ogr -f "GeoJSON" county.geo.json ../raw/county/tl_2020_us_county.shp tl_2020_us_county
ogr2ogr -f "GeoJSON" zcta5.geo.json ../raw/zcta5/tl_2020_us_zcta510.shp tl_2020_us_zcta510
cp ../raw/hrr/HRR_Bdry__AK_HI_unmodified/hrr-geojson/Hrr98Bdry_AK_HI_unmodified.geojson ./hrr.geo.json
cp ../raw/hsa/HSA_Bdry__AK_HI_unmodified/hsa-geojson/HsaBdry_AK_HI_unmodified.geojson ./hsa.geo.json

Then to convert the GeoJSON to TopoJSON:

npm install topojson ndjson-cli -g
rm -rf topojson && mkdir -p topojson
geo2topo geojson/state.geo.json > ./topojson/state.topo.json
geo2topo geojson/county.geo.json > ./topojson/county.topo.json
#geo2topo geojson/zcta5.geo.json > ./topojson/zcta5.topo.json
geo2topo geojson/hrr.geo.json > ./topojson/hrr.topo.json
geo2topo geojson/hsa.geo.json > ./topojson/hsa.topo.json

Things got stuck trying to convert the geojson to topojson for zcta5:

$ geo2topo geojson/zcta5.geo.json > ./topojson/zcta5.topo.json
node:buffer:789
    return this.utf8Slice(0, this.length);
                ^

Error: Cannot create a string longer than 0x1fffffe8 characters
    at Buffer.toString (node:buffer:789:17)
    at JSON.parse (<anonymous>)
    at ReadStream.<anonymous> (/usr/local/lib/node_modules/topojson/node_modules/topojson-server/bin/geo2topo:107:46)
    at ReadStream.emit (node:events:537:28)
    at endReadableNT (node:internal/streams/readable:1359:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  code: 'ERR_STRING_TOO_LONG'
}

Node.js v18.4.0

I tried to resolve by setting environment variable NODE_OPTIONS=--max_old_space_size=16384, but that did not fix. There is a possible solution using ndjson-cli, but I could not easily get it working.

I do not have time to try to troubleshoot this more and no longer maintain this repo, but if someone has a solution happy to implement.