US-EPA-CAMD / easey-ui

Project Management repo for EPA Clean Air Markets Division (CAMD) Business Suite of applications
MIT License
0 stars 0 forks source link

Implementation of Configuration Manager #5995

Open esaber76 opened 8 months ago

esaber76 commented 8 months ago

Unit Stack Configuration Editing Requirements

There is not currently a way to add or end stack/pipe relationships.

After the changes above have been made, display the following and allow the user to accept or reject the changes.

Note: Right now in 2.0, users are unable to delete records and would need to revert to official and start over.

Interface:

Other considerations:

Also Read the Following Comments

annalbrecht commented 4 months ago

Need screenshots from 1.0. What are the mechanics for doing the changes. Scott and Dwayne will meet to discuss further.

djw4erg commented 3 months ago

Configuration Management Principles

  1. Units (affected by a program) should belong in an MP for each quarter from the quarter of its earliest Monitor Method, through the quarter of its retirement based on UNIT_OP_STATUS (if it has retired).
  2. A unit that has never been associated with a stack or pipe in the UNIT_STACK_CONFIGURATION table will only exist in one MP as the sole location. The begin quarter of the MP will be the quarter of the earliest Monitor Method, and the end quarter will be the quarter of the unit retirement if it has retried, otherwise it will be null.
  3. A unit that has been associated with stacks or pipes will have one or more MP. These MP will contain at least one unit and one stack or pipe, but at least three locations with at least two being stacks or pipes or units (see Note 1 below). A MP's locations are connected to each other through a web of UNIT_STACK_CONFIGURATION rows with an intersection of Begin and End Dates in quarters that match the MP Begin and End Quarters.
  4. MP Reporting Frequency (MONITOR_PLAN_REPORTING_FREQ) rows indicate whether a MP reports annually (quarterly) or just for ozone season. The reporting frequency for an MP depends on whether the affected programs at units in the MP are annual or ozone season reports. The reporting frequency is always annual when annual programs exist. When only ozone season programs exist, the reporting frequency is ozone season. As the mix of annual vs ozone season programs changes for a MP, the MP can have different reporting frequencies.

Notes:

  1. MP contain at least three locations because information is reported at a unit when it only has one stack that is not connected to other units, and has zero pipes or one pipe that is not connected to other units. Under these conditions, the source does not include the stack or pipe as separate locations, and reports their data at the unit.

Unit Stack Configuration Editing/Import Rules

Note that changes refer to differences compared with the Official (CAMDECMPS) USC, not the last USC saved in the Workspace (CAMDECMPSWKS).

Updating an Existing (in Official) USC:

  1. Cannot update Begin Date
  2. Cannot change an existing End Date
  3. Can change a null End Date to an actual date but the date must be on or after the last day of the quarter of the last submitted emission report for the USC unit and stack/pipe.

Adding a New USC (compared to Official):

  1. The Begin Date must be after the quarter of the last emission report submitted for the USC unit, stack or pipe.
  2. The End Date must be either null, or on or after the Begin Date.

USC Affects on MP Notes:

  1. Changes to existing (in Official) MP should only include adding End Quarters to MP with null End Quarters.
  2. The Begin Quarters for new MP (for Official) will not include quarters for which emissions have been submitted for any location in the MP.
  3. The source will need to contact ECMPS Support to make any additional MP changes.

Single Unit MP and Method Updates

  1. For a single unit MP with a unit that was not associated with an earlier MP, the MP begin quarter is based on the earliest Monitor Method. As Monitor Methods are added, deleted or updated, if the quarter of the earliest method changes, the begin quarter of the earliest MP for the unit should change if it is a single-unit MP.

Explanatory Pseudo Implementation

Get UNIT_STACK_CONFIGURATION (USC) and UNIT data to combine into Monitoring Plans:

  1. Create a temporary Working Table with Unit, Stack-Pipe, Begin Year, Begin Quarter, End Year, End Quarter, and Identifier Key columns. The Identifier Key column should contain a sequential integer value.
  2. Add rows into the Working Table from all rows for units associated with the plant (facility) and with a Stack (Pipe) Name that begins with "C", indicating Common Stacks or Pipes. Set the begin and end years and quarters based on the USC Begin and End Dates. The End Year and Quarter should be null if the USC End Date is null.
  3. Add rows into the Working Table from all rows for units associated with the plant (facility) and with a Stack (Pipe) Name that begins with "M", indicating only Multiple Stacks since Multiple Pipes do not exist. Set the begin and end years and quarters based on the USC Begin and End Dates. The End Year and Quarter should be null if the USC End Date is null.
  4. Add rows into the Working Table for all units for the Plant. Set Stack-Pipe set to null, and Begin Year and Quarter to the quarter of the earliest Begin Date of the MONITOR_METHOD rows for the unit, Set the End Year and Quarter to the quarter of the UNIT_OP_STATUS "RET" row for the unit, if it exists, otherwise set it to null.
  5. Reference the comment Models for Adding to Working Table,which has SQL for the previous three steps.
  6. Merge the rows in Working Table using the logic in the Merge Working Table Rows comment.
  7. Match the Merged Working Table rows to existing MP on locations and begin and end quarters. For older MP/Merged Rows matches should exist because of the restrictions on Unit Stack Configuration updates. The rows that do not match will have one of two differences. For one, the locations and begin quarter will match, the MP end quarter will be null, and the merged end quarter will exist. In this end quarter of the matching MP should change to the merged end quarter. Otherwise the merged should represent a new MP.
  8. For MP that will be ended or new MP, use the logic in Updating Reporting Frequencies to update Reporting Frequency rows.
djw4erg commented 3 months ago

Models for Adding to Working Table

Note that the queries should run in the following order and for a specific plant (facility) to update the Working Table.

Common Stacks and Pipes

select  fac.Oris_Code,
        fac.Facility_Name,
        extract( year from usc.Begin_Date ) as Begin_Year,
        extract( quarter from usc.Begin_Date ) as Begin_Quarter,
        extract( year from usc.End_Date ) as End_Year,
        extract( quarter from usc.End_Date ) as End_Quarter,
        ( stp.Stack_Name || ', ' || string_agg( unt.Unitid, ', ' order by unt.Unitid ) ) as Location_List
  from  camd.PLANT fac
        join camd.UNIT unt
          on unt.Fac_Id = fac.Fac_Id 
        join camdecmpswks.UNIT_STACK_CONFIGURATION usc 
          on usc.Unit_Id = unt.Unit_Id
        join camdecmpswks.STACK_PIPE stp
          on stp.Stack_Pipe_Id = usc.Stack_Pipe_Id
         and stp.Stack_Name like 'C%'
 where  fac.Oris_Code in ( 3, 641, 708, 994, 1356, 1893, 2866, 2876, 2878, 3407, 10307, 52130, 55284 )
 group 
    by  fac.Oris_Code,
        fac.Facility_Name,
        extract( year from usc.Begin_Date ),
        extract( quarter from usc.Begin_Date ),
        extract( year from usc.End_Date ),
        extract( quarter from usc.End_Date ),
        stp.Stack_Name
 order 
    by  Oris_Code,
        Stack_Name,
        Begin_Year,
        Begin_Quarter,
        End_Year,
        End_Quarter;

Multiple Stacks (and Pipes)

select  fac.Oris_Code,
        fac.Facility_Name,
        extract( year from usc.Begin_Date ) as Begin_Year,
        extract( quarter from usc.Begin_Date ) as Begin_Quarter,
        extract( year from usc.End_Date ) as End_Year,
        extract( quarter from usc.End_Date ) as End_Quarter,
        ( string_agg( stp.Stack_Name, ', ' order by stp.Stack_Name ) || unt.Unitid || ', ' ) as Location_List
  from  camd.PLANT fac
        join camd.UNIT unt
          on unt.Fac_Id = fac.Fac_Id 
        join camdecmpswks.UNIT_STACK_CONFIGURATION usc 
          on usc.Unit_Id = unt.Unit_Id
        join camdecmpswks.STACK_PIPE stp
          on stp.Stack_Pipe_Id = usc.Stack_Pipe_Id
         and stp.Stack_Name like 'M%'
 where  fac.Oris_Code in ( 3, 641, 708, 994, 1356, 1893, 2866, 2876, 2878, 3407, 10307, 52130, 55284 )
 group 
    by  fac.Oris_Code,
        fac.Facility_Name,
        extract( year from usc.Begin_Date ),
        extract( quarter from usc.Begin_Date ),
        extract( year from usc.End_Date ),
        extract( quarter from usc.End_Date ),
        unt.Unitid
 order 
    by  Oris_Code,
        Unitid,
        Begin_Year,
        Begin_Quarter,
        End_Year,
        End_Quarter;

Units Alone

select  lst.Oris_Code,
        lst.Facility_Name,
        extract( year from lst.Begin_Date ) as Begin_Year,
        extract( quarter from lst.Begin_Date ) as Begin_Quarter,
        extract( year from lst.End_Date ) as End_Year,
        extract( quarter from lst.End_Date ) as End_Quarter,
        lst.Location_List
  from  (
            select  fac.Oris_Code,
                    fac.Facility_Name,
                    (
                        select  max( mth.Begin_Date)
                          from  camdecmpswks.MONITOR_LOCATION loc
                                join camdecmps.MONITOR_METHOD mth
                                  on mth.Mon_Loc_Id = loc.Mon_Loc_Id
                         where  loc.Unit_Id = unt.Unit_Id
                    ) as Begin_Date,
                    (
                        select  min( uos.Begin_Date)
                          from  camd.UNIT_OP_STATUS uos
                         where  uos.Unit_Id = unt.Unit_Id
                           and  uos.Op_Status_Cd = 'RET'
                    ) as End_Date,
                    unt.Unitid as Location_List
              from  camd.PLANT fac
                    join camd.UNIT unt
                      on unt.Fac_Id = fac.Fac_Id 
             where  fac.Oris_Code in ( 3, 641, 708, 994, 1356, 1893, 2866, 2876, 2878, 3407, 10307, 52130, 55284 )
        ) lst
 order 
    by  Oris_Code,
        Location_List,
        Begin_Year,
        Begin_Quarter,
        End_Year,
        End_Quarter;
djw4erg commented 3 months ago

Merge Working Table Rows

Main Logic:

/* pseudocode is 1 based, not 0 based. */

currentCursor := workingTable ordered by IdentifierKey
currentRecord := currentCursor.Fetch

while ( currentRecord.Exists )

    compareCursor := workingTable where ( IdentifierKey > currentRecord.IdentifierKey ) ordered by IdentifierKey
    compareRecord := compareCursor.Fetch

    mergeOccurred := false

    while ( compareRecord.Exists ) and ( not mergeOccurred )

        if ( currentRecord.Locations ∩ compareRecord.Locations ≠ Ø ) and ( currentRecord.Period ∩ compareRecord.Period ≠ Ø )

            { Add workingTable rows based on merge logic, which will replace the currentRecord and compareRecord rows with the next IdentifierKeys  }

            workingTable.Delete( compareRecord.IdentifierKey )
            workingTable.Delete( currentRecord.IdentifierKey )

            mergeOccurred := true

        else
            /* Get the next compare row if a merge did not occur. */
            compareRecord := compareCursor.Fetch
        end if

    end while

    if ( not mergeOccurred )
        /* Current was not merged so move to the next row as the current row. */
        currentRecord := currentCursor.Fetch
    else
        /* Recreate the currentCursor to account for deletions and inserts, and start with the first entry */
        currentCursor := workingTable ordered by IdentifierKey
        currentRecord := currentCursor.Fetch
    end if

end while

Merge Logic

combinedLocations = currentRecord.Locations ∪ compareRecord.Locations

case

    when ( compareRecord.Period.Begin = currentRecord.Period.Begin ) and ( compareRecord.Period.End = currentRecord.Period.End )

        workingTable <= [ combinedLocations, currentRecord.Period.Begin, currentRecord.Period.End ]

    when ( compareRecord.Period.Begin = currentRecord.Period.Begin ) and ( compareRecord.Period.End < currentRecord.Period.End )

        if ( currentRecord.Locations = compareRecord.Locations )
            workingTable <= [ currentRecord.Locations, currentRecord.Period.Begin, currentRecord.Period.End ]
        else
            workingTable <= [ combinedLocations, compareRecord.Period.Begin, compareRecord.Period.End ]
            workingTable <= [ currentRecord.Locations, compareRecord.Period.End + 1, currentRecord.Period.End ]
        end if

    when ( compareRecord.Period.Begin = currentRecord.Period.Begin ) and ( compareRecord.Period.End > currentRecord.Period.End )

        if ( currentRecord.Locations = compareRecord.Locations )
            workingTable <= [ compareRecord.Locations, compareRecord.Period.Begin, compareRecord.Period.End ]
        else        
            workingTable <= [ combinedLocations, currentRecord.Period.Begin, currentRecord.Period.End ]
            workingTable <= [ compareRecord.Locations, currentRecord.Period.End + 1, compareRecord.Period.End ]
        end if

    when ( compareRecord.Period.Begin < currentRecord.Period.Begin ) and ( compareRecord.Period.End = currentRecord.Period.End )

        if ( currentRecord.Locations = compareRecord.Locations )
            workingTable <= [ compareRecord.Locations, compareRecord.Period.Begin, compareRecord.Period.End ]
        else        
            workingTable <= [ compareRecord.Locations, compareRecord.Period.Begin, currentRecord.Period.Begin - 1 ]
            workingTable <= [ combinedLocations, currentRecord.Period.Begin, currentRecord.Period.End ]
        end if

    when ( compareRecord.Period.Begin > currentRecord.Period.Begin ) and ( compareRecord.Period.End = currentRecord.Period.End )

        if ( currentRecord.Locations = compareRecord.Locations )
            workingTable <= [ currentRecord.Locations, currentRecord.Period.Begin, currentRecord.Period.End ]
        else        
            workingTable <= [ currentRecord.Locations, currentRecord.Period.Begin, compareRecord.Period.Begin - 1 ]
            workingTable <= [ combinedLocations, compareRecord.Period.Begin, compareRecord.Period.End ]
        end if

    when ( compareRecord.Period.Begin < currentRecord.Period.Begin ) and ( compareRecord.Period.End < currentRecord.Period.End )

        case
            when ( compareRecord.Locations = combinedLocations ) and ( currentRecord.Locations = combinedLocations )
                workingTable <= [ combinedLocations, compareRecord.Period.Begin, currentRecord.Period.End ]
            when ( compareRecord.Locations = combinedLocations )
                workingTable <= [ compareRecord.Locations, compareRecord.Period.Begin, compareRecord.Period.End ]
                workingTable <= [ currentRecord.Locations, compareRecord.Period.End + 1, currentRecord.Period.End ]
            when ( currentRecord.Locations = combinedLocations )
                workingTable <= [ compareRecord.Locations, compareRecord.Period.Begin, currentRecord.Period.Begin - 1 ]
                workingTable <= [ currentRecord.Locations, currentRecord.Period.Begin, currentRecord.Period.End ]
            else
                workingTable <= [ compareRecord.Locations, compareRecord.Period.Begin, currentRecord.Period.Begin - 1 ]
                workingTable <= [ combinedLocations, currentRecord.Period.Begin, compareRecord.Period.End ]
                workingTable <= [ currentRecord.Locations, compareRecord.Period.End + 1, currentRecord.Period.End ]
        end case

    when ( compareRecord.Period.Begin > currentRecord.Period.Begin ) and ( compareRecord.Period.End > currentRecord.Period.End )

        case
            when ( currentRecord.Locations = combinedLocations ) and ( compareRecord.Locations = combinedLocations )
                workingTable <= [ combinedLocations, currentRecord.Period.Begin, compareRecord.Period.End ]
            when ( currentRecord.Locations = combinedLocations )
                workingTable <= [ currentRecord.Locations, currentRecord.Period.Begin, currentRecord.Period.End ]
                workingTable <= [ compareRecord.Locations, currentRecord.Period.End + 1, compareRecord.Period.End ]
            when ( compareRecord.Locations = combinedLocations )
                workingTable <= [ currentRecord.Locations, currentRecord.Period.Begin, compareRecord.Period.Begin - 1 ]
                workingTable <= [ compareRecord.Locations, compareRecord.Period.Begin, compareRecord.Period.End ]
            else
                workingTable <= [ currentRecord.Locations, currentRecord.Period.Begin, compareRecord.Period.Begin - 1 ]
                workingTable <= [ combinedLocations, compareRecord.Period.Begin, currentRecord.Period.End ]
                workingTable <= [ compareRecord.Locations, currentRecord.Period.End + 1, compareRecord.Period.End ]
        end case

    when ( compareRecord.Period.Begin < currentRecord.Period.Begin ) and ( compareRecord.Period.End > currentRecord.Period.End )

        if ( compareRecord.Locations = combinedLocations )
            workingTable <= [ compareRecord.Locations, compareRecord.Period.Begin, compareRecord.Period.End ] 
        else
            workingTable <= [ compareRecord.Locations, compareRecord.Period.Begin, currentRecord.Period.Begin - 1 ]
            workingTable <= [ combinedLocations, currentRecord.Period.Begin, currentRecord.Period.End ]
            workingTable <= [ compareRecord.Locations, currentRecord.Period.End + 1, compareRecord.Period.End ]
        end if

    when ( compareRecord.Period.Begin > currentRecord.Period.Begin ) and ( compareRecord.Period.End < currentRecord.Period.End )

        if ( currentRecord.Locations = combinedLocations )
            workingTable <= [ currentRecord.Locations, currentRecord.Period.Begin, currentRecord.Period.End ] 
        else
            workingTable <= [ currentRecord.Locations, currentRecord.Period.Begin, compareRecord.Period.Begin - 1 ]
            workingTable <= [ combinedLocations, compareRecord.Period.Begin, compareRecord.Period.End ]
            workingTable <= [ currentRecord.Locations, compareRecord.Period.End + 1, currentRecord.Period.End ]
        end if

end case
djw4erg commented 2 months ago

Updating Reporting Frequencies

Existing "Ending" MP

  1. Note: the End Quarter for the MP must be null before setting an actual End Quarter value.
  2. Delete any Reporting Frequency rows for the MP with a Begin Quarter after the new MP End Quarter.
  3. Set the End Quarter of the latest remaining Reporting Frequency for the MP to the MP End Quarter.

New MP Reporting Frequency Rules

  1. Whether an MP has an annual or OS reporting frequency depends on whether and when annual and OS programs are active for units in the MP.
  2. The active range from the programs depends on the quarters of the Unit Program Unit Monitor Cert Begin (UMCB) Date and End Date for units in the MP. The dates are in the CAMD.UNIT_PROGRAM table and each unit in the MP may be affected by multiple programs.
  3. When an annual program is active for the lifespan of the MP, create an annual MP Reporting Frequency with the same begin and end quarters as the MP, and a Reporting Frequency Code of 'Q'.
  4. When an annual program is not active during the quarter span of the MP, but an OS program is active for the lifespan of the MP, create an OS MP Reporting Frequency with the same begin and end quarters as the MP, and a Reporting Frequency Code of 'OS'..
  5. When neither an annual nor ozone season program is active for part of the period that a MP is active, that period defaults to the annual frequency of 'Q'.
  6. For any period were at least one annual program is active, the period will have the annual frequency of 'Q'.
  7. For any period that only has ozone season programs active, the period will have the ozone season frequency 0f 'OS'.
  8. Note: The above logic differs from ECMPS 1.0.

Example Table

Although this example uses two units at a common stack, it could be unit program rows at a single unit, or the combinations of a number of unit program rows at a number of units.

MP Begin MP End U1 UMCB U1 End U1 OS Ind U2 UMCB U2 End U2 OS Ind Freq 1 Freq 2 Freq 3 Notes
2022 Q1 1/1/2022 0 2022-01-01 0 Q: 2022 Q1->
2022 Q1 2023 Q3 1/1/2022 0 2022-01-01 0 Q: 2022 Q1- 2023 Q3
2022 Q1 3/31/2022 0 2022-03-31 0 Q: 2022 Q1->
2022 Q1 1/1/2022 0 2022-04-01 0 Q: 2022 Q1->
2022 Q1 4/1/2022 0 2022-04-01 0 Q: 2022 Q1-> 1
2022 Q1 1/1/2022 1 2022-01-01 1 OS: 2022 Q1->
2022 Q1 2023 Q3 1/1/2022 1 2022-01-01 1 OS: 2022 Q1- 2023 Q3
2022 Q1 3/31/2022 1 2022-03-31 1 OS: 2022 Q1->
2022 Q1 1/1/2022 1 2022-05-01 1 OS: 2022 Q1->
2022 Q1 4/1/2022 1 2022-05-01 1 Q: 2022 Q1 - 2022 Q1 OS: 2022 Q2 -> 1
2022 Q2 5/1/2022 1 2022-07-01 1 Q: 2022 Q1 - 2022 Q1 OS: 2022 Q2 ->
2022 Q2 5/1/2022 1 2022-07-01 0 Q: 2022 Q1 - 2022 Q1 OS: 2022 Q2 - 2022 Q2 Q: 2022 Q3 ->
2022 Q2 5/1/2022 0 2022-07-01 1 Q: 2022 Q1 -> 1
2022 Q2 5/1/2022 1 2022-07-01 0 Q: 2022 Q1 - 2022 Q1 OS: 2022 Q2 - 2022 Q3 Q: 2022 Q4 ->
2022 Q1 1/1/2022 1 2022-09-15 2024-05-16 0 OS: 2022 Q1 - 2022 Q2 Q: 2022 Q3 - 2024 Q2 OS: 2024 Q3 ->
2022 Q1 1/1/2022 0 2022-09-15 2024-05-16 1 Q: 2022 Q1 ->

Notes

  1. Quarters without an active program, default to annual (Q).
djw4erg commented 2 months ago

Suggested Plants for Testing

ORIS Code Facility Name Facility Id Notes
3 Barry 1 Variety of MP
641 Gulf Clean Energy Center 126 Complex Changes
708 Hammond 149 Complex Changes
994 IPL - Petersburg Generating Station 191 Complex Changes
1356 Ghent 272 Problematic in ECMPS 1.0
1893 Boswell Energy Center 372 Common Stack Created and Disbanded
2866 W H Sammis 554 Many Changes
2876 Kyger Creek 558 Split Common Stacks
2878 Bay Shore 560 Retire Common Stack and Split Unit
3407 Kingston 650 Complex with CS Combine
6090 Sherburne County 814 Retired Unit
6264 Mountaineer (1301) 860 Annual and OS Only Units
10307 Bellingham 1092 Common Pipe
52130 BP Whiting Business Unit 7699 Common Pipe (Inactive)
55284 Big Sandy Peaker Plant 1398 Commom Pipe Change
djw4erg commented 2 months ago

Affected and Needed Tables

Affected

The tables exist in both the CAMDECMPS (Official) and CAMDECMPSWKS (Workspace). The changes occur in the Workspace versions and are submitted to the Official versions.

Needed to determine quarter range for which unit should belong to an MP

Needed to determine the latest emission submitted quarter for a unit and stack/pipe

  1. MONITOR_LOCATION (CAMDECMPS): Matched on UNIT_ID or STACK_PIPE_ID
  2. MONITOR_PLAN_LOCATION (CAMDECMPS): Matched on MON_LOC_ID
  3. EMISSION_EVALUATION (CAMDECMPS): Matched on MON_PLAN_ID
  4. REPORTING_PERIOD (CAMDECMPSMD): Matched on RPT_PERIOD_ID and used to determine the latest quarter.
djw4erg commented 1 month ago

Adding MP for Units without an MP

  1. Add section to Configuration Management to display units that are not currently in a MP.
  2. Allow the user to select from these units to create MP and set the anticipated begin quarter for the MP.
  3. Saving the created MP should result in the creation of the following:
    • MONITOR_PLAN row including a BEGIN_RPT_PERIOD_ID corresponding to the anticipated begin quarter.
    • MONITOR_PLAN_LOCATION row with the MON_PLAN_ID for the created MONITOR_PLAN row, and MON_LOC_ID for the unit.