Closed bendnorman closed 2 years ago
I bet could write a little script / function that did this update so long as nothing had changed, and flagged it for human attention when something has changed.
I think you already did! Just found this notebook that compares consecutive years in the f1_row_lit_tbl
table. I reran it and it looks like nothing changed between 2019 and 2020.
Would you like to include the logic in the notebook in the ferc1 extract step? I think keeping it in the notebook is sufficient given we only do this once a year.
Ah, right I vaguely remember that notebook. Probably not worth changing anything, since they're switching to XBRL next year anyway. So we get to build a whole brand new ETL process!
The ferc1 rows were checked and are in the 2020
branch.
Update the FERC 1 row maps for individual tables, based on the row_num associated with each named field in the f1_row_lit_tbl -- if the row_chg_yr column hasn’t changed, you should be able to copy the previous year’s column over to the new year.