NYCPlanning / db-safegraph

SafeGraph ETL
4 stars 1 forks source link

Change output paths to include 'dev/production/staging' #91

Closed mgraber closed 3 years ago

mgraber commented 3 years ago

82

For example:

aws.execute_query(
        query=query, 
        database="safegraph", 
        output="output/production/social_distancing/device_counts_by_county.csv.zip"
    )

or

for year_qrtr, range in quarters.items():
    start = range[0]
    end = range[1]
    print(year_qrtr, start, end) 
    aws.execute_query(
        query=query.format(start, end, poi_latest_date), 
        database="safegraph", 
        output=f"output/dev/poi/daily_zip_poivisits_by_subsector/daily_zip_poivisits_by_subsector_{year_qrtr}.csv.zip"
    )
SPTKL commented 3 years ago

also need to change the jupyter notebook to reflect the change