coopercenter / postgres-etl

Code for retrieving, organizing, and managing data in our postgres database
GNU General Public License v3.0
1 stars 0 forks source link

Start saving each vintage of the PJM New Services Queue data #91

Open athena-small opened 4 years ago

athena-small commented 4 years ago

@jacksonbrandberg @christinazhongg @LEEPYUNG @Mailuu96 @chloe-fauvel :

The PJM New Services Queue provides data only for the current state of the queue. When a project's status is updated, apparently the record of its history is lost.

The task: We want to keep track of each update in the status and approval stage of each project. We want to create a data structure and retrieval code that allows us to track these transitions in status, with dates.

This does not mean we need to store necessarily a full copy of each day's version of the queue. What we want is a dataset that records each change in status, with the date of change, for each project in the queue.

One of the uses of these data will be to estimate success probabilities and expected time-to-completion for each project in the queue. These probabilities will be very useful for estimating the real (as opposed to notional) pipeline of future generation capacity, for different fuel types. We need data on status transitions to estimate those probabilities.