Add Submission Validation Checks

[djw4erg]

Overview

The purpose of this ticket is to ensure that needed submission validation checks occur before files are submitted. This includes checks on:

• User Permissions
• User Certification Statements Signed
• User Submitter Role
• File Submission Criteria

Original Need

Originally this ticket was focused on the File Submission Criteria and was intended to duplicate checking that (should) already occur in the user interface in the "submission" end point(s). The purpose of this duplication was to ensure that the submission endpoints directly performed the verification needed for submissions before performing the submissions. At the time this was needed in case users directly used the endpoints to perform a submission instead of using the provided endpoints.

However, to perform a submission the users of the "submission" endpoints need a bearer token. With the implementation of ICAM a user can only get a bearer token with a call from an EASEY URL. As a result, an absolute need does not exist to perform checking in the "submission" endpoints that already occurs in the UI before calling the endpoints.

New Understanding

• User Permissions
  • Add check for missing user permissions to the "camd-services/submission/queue" endpoint.
  • The endpoint should check to ensure that the user has permission to submit:
  • The MP (if being submitted),
  • The QAT for each QAT id.
  • The QCE for each QCE id.
  • The TEE for each TEE id.
  • The EM Report for the MP and each reporting period id.
• User Certification Statements Signed
  • Add missing check for unsigned certification statements to the "camd-services/submission/queue" endpoint.
  • The CBS permissions API will be updated (see https://camddevelopment.atlassian.net/browse/EASEY-420) to contain an additional "missingCertificationStatements" element. If "missingCertificationStatements" is "true" then the submission should fail.
  • Sample API Responses
  • Preparer with unsigned cert statements
    • {"plantList":[{"facId":10697,"orisCode":999901,"permissions":[]}],"missingCertificationStatements":true}
  • Preparer without unsigned cert statements
    • "plantList":[{"facId":10696,"orisCode":999901,"permissions":[]}],"missingCertificationStatements":false}
  • Submitter without unsigned cert statements
    • {"plantList":[{"facId":10695,"orisCode":990003,"permissions": "DSMP","DSQA","DSEM"]}],"missingCertificationStatements":false}
• User Submitter Role
  • This check occurs as a byproduct of the call to "auth-mgmt/sign/create-activity" endpoint.
• File Submission Criteria
  • Add check for missing file criteria to the "camd-services/submission/queue" endpoint.
  • These are or should be duplicate checks, but should occur to ensure that submission criteria (like submission window is open) have not changed while users are working through the submission screens.

General Notes

1. ECMPS 1.0 has PENDING_STATUS_CD columns in the MONITOR_PLAN (MP), EMISSION_EVALUATION (EM), QA_SUPP_DATA (QAT), QA_CERT_EVENT (QCE), and TEST_EXTENSION_EXEMPTION (TEE) tables. The column usually contains nulls, but contains 'PENDING' or 'PENDUP' when the data has been submitted but has not been loaded into the actual data tables.
   1. The 'PENDING' or 'PENDUP' statuses are akin to having started a submission job that has not completed. However, ECMPS 2.0 does not appear to set these statuses.
2. ECMPS 1.0 has an IN_SYNCH_FLG column in the client SYNCHRONIZATION_MANAGEMENT table that indicates whether the client version of the table matches the host version. The SYNCHRONIZATION_MANAGEMENT table is used to determine whether data associated with specific monitoring plans in individual client ECMPS databases need to be updated. In ECMPS 2.0, the number of client databases are akin to the lone CAMDECMPSWKS schema.
   1. Because only one source of submissions exists, the need to check for being in-synch no longer exists.

User Conditions

1. Certification Statement Check
   • A user cannot submit if they have unsigned certification statements.
   • CBS permissions API will return an additional value in the JSON for the cert statement status
2. Permission Check
   • What should happen if the user can submit some of but not all of the files in the submission set?
     • I believe that the set should be failed.
   • CBS permissions API will return a permissions list.

Notes about Add Certification Statement Status to Permissions API Ticket

1. The ticket will include a flag containing the user’s certification statement status in the response.
2. If mocked permissions are set to be on, use the contents of the mocked permissions file instead of making the API call.
3. Ensure that #6344 is completed first.

File Submission Criteria

Monitoring Plan (MP) Submission Conditions

1. MONITOR_PLAN.SUBMISSION_AVAILABILITY_CD equals 'REQUIRE' or MONITOR_PLAN.UPDATED_STATUS_FLG equals 'Y'.
2. MONITOR_PLAN.NEEDS_EVAL_FLG equal 'N'.
3. Linked CHECK_SESSION.SEVERITY_CD is not null (or missing row) and not 'FATAL'.
   1. 'CRIT1' submissions are allowed, but the handling of 'CRIT1', which should already exist, should be different.

Emission Report (EM) Submission Conditions

1. EMISSION_EVALUATION SUBMISSION_AVAILABILITY_CD equals 'REQUIRE', or SUBMISSION_AVAILABILITY_CD equals 'GRANTED' and UPDATED_STATUS_FLG equals 'Y'.
2. EMISSION_EVALUATION.NEEDS_EVAL_FLG equal 'N'.
3. Linked CHECK_SESSION.SEVERITY_CD is not null (or missing row) and not 'FATAL'.
   1. 'CRIT1' submissions are allowed, but the handling of 'CRIT1', which should already exist, should be different.

QA Test (QAT) Submission Conditions

1. Related QA_SUPP_DATA.SUBMISSION_AVAILABILITY_CD equals 'REQUIRE', or QA_SUPP_DATA.SUBMISSION_AVAILABILITY_CD equals 'GRANTED' and TEST_SUMMARY.UPDATED_STATUS_FLG equals 'Y', or QA_SUPP_DATA row does not exist and TEST_SUMMARY.UPDATED_STATUS_FLG equals 'Y'.
2. TEST_SUMMARY.NEEDS_EVAL_FLG equal 'N'.
3. Linked CHECK_SESSION.SEVERITY_CD is not null (or missing row) and not 'FATAL'.
   1. 'CRIT1' submissions are allowed, but the handling of 'CRIT1', which should already exist, should be different.

QA Certification Event (QCE) Submission Conditions

1. QA_CERT_EVENT.SUBMISSION_AVAILABILITY_CD equals 'REQUIRE' or QA_CERT_EVENT.UPDATED_STATUS_FLG equals 'Y'.
2. QA_CERT_EVENT.NEEDS_EVAL_FLG equal 'N'.
3. Linked CHECK_SESSION.SEVERITY_CD is not null (or missing row) and not 'FATAL'.
   1. 'CRIT1' submissions are allowed, but the handling of 'CRIT1', which should already exist, should be different.

Test Extension and Exemption (TEE) Submission Conditions

1. TEST_EXTENSION_EXEMPTION.SUBMISSION_AVAILABILITY_CD equals 'REQUIRE' or TEST_EXTENSION_EXEMPTION.UPDATED_STATUS_FLG equals 'Y'.
2. TEST_EXTENSION_EXEMPTION.NEEDS_EVAL_FLG equal 'N'.
3. Linked CHECK_SESSION.SEVERITY_CD is not null (or missing row) and not 'FATAL'.
   1. 'CRIT1' submissions are allowed, but the handling of 'CRIT1', which should already exist, should be different.

Acceptance Tests

1. If the user attempts a submission for file types they no longer have permission to submit , the following error message should be returned in ECMPS: "You are no longer permitted to submit the following files: [list of files no longer permitted]. Please use the CAMD Business System to verify your representative or agent responsibilities."
   • The submission set should not go through if the user is unable to submit of the files in the set (ex. user can submit MP and QA data but not EM)
2. If the user has not signed all their certification statements in CBS, the following error message should be returned in ECMPS: "You have not signed all of the necessary certification statements which are associated with your responsibilities as a representative or agent. Until these certification statements have been signed, you will not be able to submit data via ECMPS. Please use the CAMD Business System to sign all of your required certification statements."
3. CDX Roles: When a user who does not have a CDX role that allows submission (Submitter, Sponsor or Initial Authorizer) the submission is not allowed and the user receives an appropriate message.
4. When submission criteria for any file in a submission set fails, the whole submission fails and the user receives a message indicating that the files failed submission criteria, and lists the files that failed the criteria.
   • Message: Files in the submission set do not meet the criteria for submission. The problem files include {File List}.
   • File name formats:
     • MP: List of locations in the MP.
     • EM: List of locations, Year and Quarter of the emission report.
     • QAT: Location, test number, and test type or the test.
     • QCE: Location, System and/or Component Identifiers, Cert Event Code, and Cert Event Date.
     • TEE: Location, System and/or Component Identifiers, Extension/Exemption Code, Year and Quarter.

[mark-hayward-erg]

@djw4erg Here are the user-related checks that need to happen on submit:

• Add cert statement check on submit
  • if user has unsigned cert statements, fail submit with message "...."

• Add permissions check on submit

  • if use can longer submit one more files, fail submit with message "...."
  • Call CBS permissions API prior to showing submit modal

• CBS permissions API will return an additional value in the JSON for the cert statement status

  • { plantList: [], missingCertificationStatements: false }

• if missingCertificationStatements is true, show message

  • "You have not signed all of the necessary certification statements which are associated with your responsibilities as a representative or agent. Until these certification statements have been signed, you will not be able to submit data via ECMPS. Please use the CAMD Business System to sign all of your required certification statements."

• if the user no longer has the ability to submit any of the selected files (based on the updated CBS permissions list), show message

  • "You are no longer permitted to submit the following files: [list of files no longer permitted]. Please use the CAMD Business System to verify your representative or agent responsibilities."

The changes to the CBS permissions API will be completed via https://camddevelopment.atlassian.net/browse/EASEY-420

[mark-hayward-erg]

Testing will be blocked by https://camddevelopment.atlassian.net/browse/EASEY-420