Integrate 860m

[TrentonBush]

Because the 860m data comes from PUDL, much of the ETL code will live in PUDL but some application-specific code will be in this repo. Where should that boundary be?

Output Requirements from PUDL:

• Include historical 860m back to at least Jan 2022. Deeper historical data is welcome but not a priority
• be able to describe the reported status of proposed projects with monthly resolution
• include enough project-level information to hopefully be able to link records with ISO queues:
  • resource type(s)
  • capacity(s)
  • location (county_fips would be ideal, higher resolution is OK but not necessary)
  • utility
  • point of interconnection, if available

Output Requirements from Application Code:

• output a dense table with a primary key of (project_id, timestamp) and one column project_status
• process the above into a "changelog" style wide format table with primary key of project_id and columns corresponding to various status changes, eg status_1_to_status_2, and values containing the date of that status transition.

Other Considerations: Because PUDL is a bigger and more complex project, it is slower moving: the time to create satisfactory code and get PRs through the queue can be long. For this reason, I think we should start with as much prototyping as possible in this repo until we get the right outputs. Then we can refactor/move useful transformations back upstream to PUDL without deadline pressure.

This repo will also need to be updated to use the latest PUDL database. We can also remove the dependency on pudl's codebase.

[jdangerx]

@TrentonBush we had mentioned wanting to split this up into the two major output tables, do you still think that's a good idea?

[jdangerx]

Sounds like this is mostly sorted, though there are still some questions about the output schema. Once those are resolved, should we close this issue @TrentonBush ?