add hpd historical data

damonmcc commented 1 year ago

goals

be able to archive new source data: HPD Historical Units by Project (hpd_historical_units_by_building)

changes

add template file and python script
change readme to have a more common example of library archive arguments

notes

emulates approach for similar source data hpd_hny_units_by_building
screenshot of archived data in edm-recipes S3 bucket:
successful use of the archived data in a downstream process

alexrichey commented 1 year ago

@damonmcc (and cc @fvankrieken ) This all looks fine, but I suppose I don't fully understand the rationale for the inheritance pattern in the ./library/scripts folder. Looks like there's a ton of boilerplate... for example, why not just implement

    def runner(self) -> str:
        df = self.ingest()
        local_path = df_to_tempfile(df)
        return local_path

in a parent class, since those lines are reproduced all over. Seems like you could eliminate quite a few of these files.

And in the case of the script added here, what about having sheetname defined in the template, and have an ExcelScriptorInterface or something? Seems more traditionally OOP'y, but let me know if there's something I don't understand.

Maybe a bigger discussion. Otherwise LGTM!

alexrichey commented 1 year ago

06_20_2023_Lookback Legislation.xlsx also wasn't sure where this file comes from

NYCPlanning / db-data-library