Update MP Import Checks to Skip "Inactive" Records

Overview

Background

ECMPS 2.0 used a number of ECMPS 1.0 screen checks as DTO or Check Service checks. In ECMPS 1.0 a user can ignore non-Fatal results of screen checks and successfully save the data. Additionally screen checks are not applied to MP XML imports. However, most of the screen checks are also report (submission) checks that will prevent the submission of a MP with Fatal or Critical 1 errors with one general exception. Some of the checks are not run against inactive data when run as report checks. For these checks it is possible to submit MP data that is inactive that would have failed if it was active.

ECMPS 2.0's implementation of checks as DTO or Check Service checks causes problems when the same checks in ECMPS 1.0 are skipped for inactive data. When exported from ECMPS 2.0, reimporting a MP with skipped Critical 1 results in ECMPS 1.0 for inactive data currently results in a failed import. Those check results should not occur for inactive data in ECMPS 2.0.

Although, there are thirty-six Critical 1 check results implemented in ECMPS 2.0 that could exhibit a problem in ECMPS 2.0, the majority (see this) do not currently (September 20, 2024) have any submitted rows that would be problematic. As a result, this ticket will only include the check results that currently have problem rows.

Ticket generated from #5942.

Inactive MP Record Definition

The Record Begin Date / End Date and the Evaluation Begin Date / End Date do not overlap. Basically, Record Begin Date is after Evaluation End Date or Record End Date is before Evaluation Begin Date. A null Record End Date is always after the Evaluation Begin Date. Evaluation Begin Date is always 1/1/2008. Evaluation End Date is the Current Date plus a year.

This definition applies to Analyzer Range, Default, Formula, Load, Location, Method, Qualification, Rectangular Duct WAF, Span, System, System Component, System Fuel Flow, Unit Capacity, and Unit Fuel data.

The definition also applies to Unit Control data except that it's Install Date and Retire Date instead of Begin Date and End Date respectively.

For Component data the definition is related to the definition above, but a Component is inactive when no related System Component Begin Date / End Date overlaps with Evaluation Begin Date / End Date.

Requirement

Alter the check implementations listed in Affected Check Results to only return the result if the record being checked is "not inactive" according to the definition above.

Any other unrelated checking should still occur. For DTO checking investigate using DTO Custom validation classes instead of Conditional Validation
The following Default checks, listed in Affected Checks, implemented a version of this change that used the "has End Date" definition for an inactive record. They will need to be corrected.
- DEFAULT-51-A
- DEFAULT-52-A
- DEFAULT-53-A

Affected Check Results

For the "Type" Column

CS indicates the Check Services in the Monitor Plan API.
DTO indicates the Data Transfer Object checking for the Monitor Plan API.

Result Code	Result Description	Type
COMPON-13-A	Component Sample Acquisition Method Code is not in the list of valid values.	CS
COMPON-14-A	Component Basis Code not reported.	CS
DEFAULT-51-A	Default Purpose Code not reported.	DTO
DEFAULT-52-A	Default Source Code not reported.	DTO
DEFAULT-53-A	Default Fuel Code not reported.	DTO
SPAN-3-B	For Flow Span, MPF Value is less that 500,000.	DTO
SPAN-17-A	For Flow Span, Flow Full Scale Range not reported.	CS
SPAN-17-B	For Flow Span, Flow Full Scale Range is less than the Flow Span Value.	CS
SPAN-17-C	For non-Flow Span, Flow Full Scale Range is not null.	CS
SYSTEM-7-B	Monitoring System ID does not consist of 3 alphanumeric characters.	DTO

Acceptance Criteria

For each Affected Check Result use the scripts below to identify a test case by exporting it and then importing it.

Existing Row Count Scripts

-- COMPON-13-A
select  *
  from  camdecmps.COMPONENT dat
 where  dat.Acq_Cd is null
    and dat.Component_Type_Cd not in ( 'CALR', 'DAHS', 'DL', 'DP', 'FLC', 'GCH', 'MS', 'PLC', 'PRB', 'PRES', 'TANK', 'TEMP' );

-- COMPON-14-A
select  *
  from  camdecmps.COMPONENT dat
 where  dat.Basis_Cd is null
    and dat.Component_Type_Cd in ( 'CO2', 'FLOW', 'HCL', 'HF', 'HG', 'NOX', 'O2', 'SO2', 'STRAIN' );

-- DEFAULT-51-A
select  *
  from  camdecmps.MONITOR_DEFAULT dat
 where  dat.Default_Purpose_Cd is null;

-- DEFAULT-52-A
select  *
  from  camdecmps.MONITOR_DEFAULT dat
 where  dat.Default_Source_Cd is null;

-- DEFAULT-53-A
select  *
  from  camdecmps.MONITOR_DEFAULT dat
 where  dat.Fuel_Cd is null;

-- SPAN-17-A
select  *
  from  camdecmps.MONITOR_SPAN dat
 where  dat.Flow_Full_Scale_Range is null
   and  dat.Component_Type_Cd = 'FLOW';

-- SPAN-17-B
select  *
  from  camdecmps.MONITOR_SPAN dat
 where  dat.Flow_Full_Scale_Range < dat.Span_Value
   and  dat.Component_Type_Cd = 'FLOW';

-- SPAN-17-C
select  *
  from  camdecmps.MONITOR_SPAN dat
 where  dat.Flow_Full_Scale_Range is not null
   and  dat.Component_Type_Cd != 'FLOW';

-- SYSTEM-7-B
select  *
  from  camdecmps.MONITOR_SYSTEM dat
 where  length( trim( dat.System_Identifier ) ) != 3;

US-EPA-CAMD / easey-ui