search

Nate-Wessel / urban-freight

Maps for a dashboard on GHG emissions from urban freight in Canadian cities
https://www.pembina.org/urban-freight
0 stars 0 forks source link

Fetch osm data - only what's needed #54

Closed Nate-Wessel closed 2 years ago

Nate-Wessel commented 2 years ago

This is a node.js script for fetching all the OSM data from overpass. Overpass query limits mean that it will likely throw an error after a few requests, but you can just comment out some of the IDs and rerun again after waiting a few minutes.

Total data size for all cities, as JSON: ~225MB

jamaps commented 2 years ago

I'm getting an error for Halifax and Victoria. Did these download okay for you?

fetching data for 3227127
fetching data for 2564500
fetching data for 9344588
undefined:1
<?xml version="1.0" encoding="UTF-8"?>
^

SyntaxError: Unexpected token < in JSON at position 0
    at JSON.parse (<anonymous>)
    at Response.json (file:///var/www/html/urban-freight/node_modules/node-fetch/src/body.js:149:15)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async getData (file:///var/www/html/urban-freight/src/data/data-prep-utils/fetch-osm.mjs:47:2)
    at async file:///var/www/html/urban-freight/src/data/data-prep-utils/fetch-osm.mjs:20:2
Nate-Wessel commented 2 years ago

That's the overpass query limit I mentioned. It's just throttling your use of the API after a few requests