SimpleGeo imports

[thisisaaronland]

• [x] 21849 slice-us-ak.geojson
• [x] 154142 slice-us-al.geojson
• [x] 132718 slice-us-ar.geojson
• [x] 217817 slice-us-az.geojson
• [x] 1516323 slice-us-ca.geojson
• [x] 241169 slice-us-co.geojson
• [x] 146378 slice-us-ct.geojson
• [x] 38171 slice-us-de.geojson
• [x] 842091 slice-us-fl.geojson
• [x] 474331 slice-us-ga.geojson
• [x] 50740 slice-us-hi.geojson
• [x] 132599 slice-us-ia.geojson
• [x] 55515 slice-us-id.geojson
• [x] 357331 slice-us-il.geojson
• [x] 220960 slice-us-in.geojson
• [x] 106774 slice-us-ks.geojson
• [x] 121614 slice-us-ky.geojson
• [x] 150520 slice-us-la.geojson
• [x] 318444 slice-us-ma.geojson
• [x] 228021 slice-us-md.geojson
• [x] 43665 slice-us-me.geojson
• [x] 392677 slice-us-mi.geojson
• [x] 276975 slice-us-mn.geojson
• [x] 232844 slice-us-mo.geojson
• [x] 91785 slice-us-ms.geojson
• [x] 36439 slice-us-mt.geojson
• [x] 417728 slice-us-nc.geojson
• [x] 35525 slice-us-nd.geojson
• [x] 88862 slice-us-ne.geojson
• [x] 62459 slice-us-nh.geojson
• [x] 481420 slice-us-nj.geojson
• [x] 78066 slice-us-nm.geojson
• [x] 110073 slice-us-nv.geojson
• [x] 842577 slice-us-ny.geojson
• [x] 520857 slice-us-oh.geojson
• [x] 126033 slice-us-ok.geojson
• [x] 221228 slice-us-or.geojson
• [x] 573008 slice-us-pa.geojson
• [x] 54973 slice-us-ri.geojson
• [x] 144689 slice-us-sc.geojson
• [x] 40079 slice-us-sd.geojson
• [x] 276381 slice-us-tn.geojson
• [x] 1122098 slice-us-tx.geojson
• [x] 97153 slice-us-ut.geojson
• [x] 329410 slice-us-va.geojson
• [x] 31217 slice-us-vt.geojson
• [x] 348264 slice-us-wa.geojson
• [x] 259914 slice-us-wi.geojson
• [x] 41158 slice-us-wv.geojson
• [x] 27226 slice-us-wy.geojson

(12932290 total)

[thisisaaronland]

Notes to self. None of this has been automated or scripted yet. Maybe it's time. Maybe not...

Before:

mv whosonfirst-data-venue-us-oregon whosonfirst-data-venue-us-or
cd whosonfirst-data-venue-us-or/
curl -s -o Makefile https://raw.githubusercontent.com/whosonfirst/whosonfirst-data-utils/master/make/Makefile
make concordances

Re-indexing the csvdb server should happen automagically but I have doubts it is working correctly and haven't had time to investigate so:

./bin/wof-csvdb-server -columns 'wof:id,sg:id,gp:id,gn:id,fips:code' -cors /usr/local/data/whosonfirst-data*/meta/wof-concordances-latest.csv
time to index /usr/local/data/whosonfirst-data/meta/wof-concordances-latest.csv: 6.003949431s
time to index /usr/local/data/whosonfirst-data-venue-dk/meta/wof-concordances-latest.csv: 46.508326ms
time to index /usr/local/data/whosonfirst-data-venue-us-ca/meta/wof-concordances-latest.csv: 19.13601369s
time to index /usr/local/data/whosonfirst-data-venue-us-fl/meta/wof-concordances-latest.csv: 11.700793904s
time to index /usr/local/data/whosonfirst-data-venue-us-ma/meta/wof-concordances-latest.csv: 2.778008834s
time to index /usr/local/data/whosonfirst-data-venue-us-mi/meta/wof-concordances-latest.csv: 6.844124369s
time to index /usr/local/data/whosonfirst-data-venue-us-nc/meta/wof-concordances-latest.csv: 8.324600249s
time to index /usr/local/data/whosonfirst-data-venue-us-nj/meta/wof-concordances-latest.csv: 11.73808044s
time to index /usr/local/data/whosonfirst-data-venue-us-ny/meta/wof-concordances-latest.csv: 7.261153099s
time to index /usr/local/data/whosonfirst-data-venue-us-oh/meta/wof-concordances-latest.csv: 5.490265592s
time to index /usr/local/data/whosonfirst-data-venue-us-or/meta/wof-concordances-latest.csv: 964.026139ms
time to index /usr/local/data/whosonfirst-data-venue-us-pa/meta/wof-concordances-latest.csv: 15.287061795s
time to index /usr/local/data/whosonfirst-data-venue-us-tx/meta/wof-concordances-latest.csv: 13.443723475s
time to index /usr/local/data/whosonfirst-data-venue-us-wa/meta/wof-concordances-latest.csv: 1.290512ms
wof-csvdb-server running at localhost:8228

During:

python ./import.py -a /usr/local/data/whosonfirst-data/data -d /usr/local/data/whosonfirst-data-venue-us-or/data slice-us-or.geojson

After:

cd /usr/local/data/whosonfirst-data-venue-us-or
make concordances
make sync-s3
make sync-es host=HOST