ingest new transportation data for 2023 update

NYCPlanning / db-equitable-development-tool

Data Repo for the equitable development tool (EDDT)

MIT License

0 stars 0 forks source link

ingest new transportation data for 2023 update #319

Closed damonmcc closed 1 year ago

damonmcc commented 1 year ago

motivations

part of #312
new transportation data was shared by Housing on 3/13

changes

add new transportation data
edit access_subway_and_access_ADA.py to process new data
add read_from_excel to ingestion_helpers.py
add test_ingestion_helpers.py
trigger tests on changes to ingest/**.py
don't install R in dev container

tests

PR tests passing
exported categories to edm-publishing on this branch. action runs:
- quality_of_life

fvankrieken commented 1 year ago

Commenting for posterity - motivation to move this from getting this file from S3 vs local excel is that most of this repo reads in from local files, so mainly for sake of internal consistency.

I still think in a bit, we should maybe move all sources to data library/s3. I get that it's a little annoying if they're all things we grab quite manually as it's just an extra step to put it in s3 and then grab it back, but long term I'd still prefer consistency across projects and getting all source files in edm-recipes in the same manner, rather than having some stored in github

@mbh329 I'm gonna go ahead and merge this