search

pelias / openaddresses

Pelias import pipeline for OpenAddresses.
MIT License
51 stars 43 forks source link

use npm "temp" instead of npm "tmp" #474

Closed missinglink closed 3 years ago

missinglink commented 3 years ago

use npm "temp" instead of npm "tmp".

It doesn't make much sense to have both modules when temp can do everything tmp does 🤷 The documentation for temp isn't super clear on the usage of .path() (edit: this is the relevant part), it's a bit clearer by looking at the code.

This code isn't covered by unit tests so this PR is DRAFT until I can figure out how to manually test it.

missinglink commented 3 years ago

manually tested and works fine for pelias config files containing imports.openaddresses.files and also without.

@orangejulius could you please sanity check this for me before it goes out?

PELIAS_CONFIG=/code/pelias/openaddresses/pelias.json npm run download

> pelias-openaddresses@0.0.0-development download /Users/peter/code/pelias/openaddresses
> ./bin/download

info: [openaddresses-download] Attempting to download selected data files: us/or/portland_metro.csv,us/or/city_of_salem.csv,us/or/marion_and_polk.csv,us/or/marion.csv,us/or/hood_river.csv,us/wa/city_of_richland.csv,us/wa/clark.csv
info: [openaddresses-download] Downloading us/or/portland_metro.csv
debug: [openaddresses-download] downloading https://results.openaddresses.io/latest/run/us/or/portland_metro.zip
debug: [openaddresses-download] unzipping /tmp/foo/us-or-portland_metro2020818-45800-165qivt.b5gz.zip to /tmp/foo
info: [openaddresses-download] Downloading us/or/city_of_salem.csv
debug: [openaddresses-download] downloading https://results.openaddresses.io/latest/run/us/or/city_of_salem.zip
debug: [openaddresses-download] unzipping /tmp/foo/us-or-city_of_salem2020818-45800-xwsvy1.np3cl.zip to /tmp/foo
info: [openaddresses-download] Downloading us/or/marion_and_polk.csv
debug: [openaddresses-download] downloading https://results.openaddresses.io/latest/run/us/or/marion_and_polk.zip
debug: [openaddresses-download] unzipping /tmp/foo/us-or-marion_and_polk2020818-45800-3x1wi0.p42pi.zip to /tmp/foo
info: [openaddresses-download] Downloading us/or/marion.csv
debug: [openaddresses-download] downloading https://results.openaddresses.io/latest/run/us/or/marion.zip
debug: [openaddresses-download] unzipping /tmp/foo/us-or-marion2020818-45800-ag06g4.3i006.zip to /tmp/foo
info: [openaddresses-download] Downloading us/or/hood_river.csv
debug: [openaddresses-download] downloading https://results.openaddresses.io/latest/run/us/or/hood_river.zip
debug: [openaddresses-download] unzipping /tmp/foo/us-or-hood_river2020818-45800-mp70fp.wp76c.zip to /tmp/foo
info: [openaddresses-download] Downloading us/wa/city_of_richland.csv
debug: [openaddresses-download] downloading https://results.openaddresses.io/latest/run/us/wa/city_of_richland.zip
debug: [openaddresses-download] unzipping /tmp/foo/us-wa-city_of_richland2020818-45800-18odmzv.vdn6.zip to /tmp/foo
info: [openaddresses-download] Downloading us/wa/clark.csv
debug: [openaddresses-download] downloading https://results.openaddresses.io/latest/run/us/wa/clark.zip
debug: [openaddresses-download] unzipping /tmp/foo/us-wa-clark2020818-45800-632imj.amt0d.zip to /tmp/foo
info: [openaddresses-download] All done!
l /tmp/foo
total 4.0K
drwxr-xr-x 4 peter 128 Sep 18 10:59 us
-rw------- 1 peter 111 Aug 15 15:18 README.txt
PELIAS_CONFIG=/code/pelias/openaddresses/pelias.json npm run download

> pelias-openaddresses@0.0.0-development download /Users/peter/code/pelias/openaddresses
> ./bin/download

info: [openaddresses-download] Attempting to download all data
debug: [openaddresses-download] downloading https://data.openaddresses.io/openaddr-collected-global.zip
debug: [openaddresses-download] downloading https://data.openaddresses.io/openaddr-collected-global-sa.zip
^C%
orangejulius commented 3 years ago

Yup, this works for me as well. I guess manual testing of the downloader is something we should do whenver we update dependencies?