search

US-EPA-CAMD / easey-ui

Project Management repo for EPA Clean Air Markets Division (CAMD) Business Suite of applications
MIT License
0 stars 0 forks source link

Incorporate recipient list endpoint to the submission window job #5826

Open Kyle-Herceg opened 10 months ago

Kyle-Herceg commented 10 months ago

Now that we have the recipient list endpoint from ERG, we can wire up the email queue process to populate the emails to send with the correct emails returned from the recipient list endpoint.

Changes Needed

  1. Add missing template text to the [EASEY_CONTENT] repository's Dev folder based on the location in the EMAIL_TEMPLATE table.
  2. Add environment variable EASEY_ECMPS_SUPPORT_TO_EMAIL including determining the email address to use in the Dev environment.
  3. Updated both the SubmissionReminderProcessQueue and SubmissionWindowProcessQueue Execute methods to include the email address in the EASEY_ECMPS_SUPPORT_TO_EMAIL environment variables email address as an additional To address.
  4. Change both the SubmissionReminderProcessQueue and SubmissionWindowProcessQueue Execute methods to use 'SUBMISSIONREMINDER' and 'WINDOWNOTIFICATION' respectively as the emailType instead of 'submissionReminder' and 'submissionWindow'.

Background Information

Subsequent comments made on this issue with the following headings contain background information.

Original Notes

*This work is contained in easey-quartz, and the recipient list endpoint is currently being mocked.

JanellC commented 10 months ago

On 10/23 during DSU it was discussed DPC will work on this

alangmaid commented 10 months ago

@ergjustin reach out to @mark-hayward-erg and @esaber76 before you start this. We should schedule a meeting.

annalbrecht commented 6 months ago

First, make sure the correct submission window job is running. Another meeting with Mark, Scott, and Chris should be set up to discuss this ticket before development can start.

mark-hayward-erg commented 5 months ago

Submission confirmation emails are only going to the users performing the submissions. The various ECMPS 2.0 environments need to use the CBS Recipient List API so further testing can be done for ECMPS 2.0 events and emails. The API will return recipient lists for the following actions:

New stack/pipe monitoring plan submission. Submission confirmations. Submission reminders. Windows notifications. https://camddevelopment.atlassian.net/browse/EASEY-255 includes examples of who should be in the recipient lists to help with testing efforts in 2.0.

djw4erg commented 4 months ago

Key Code Objects

init_and_close_em_submission_access

add_window_email

SubmissionReminderProcessQueue

SubmissionWindowProcessQueue

EmailQueue

MailTemplateService

djw4erg commented 4 months ago

Code Information

Submission Reminder

Submission Window

djw4erg commented 4 months ago

Currently Implemented Emailing Steps

Emission Submission Windows Initial Creation, Reminders, and Notifications (init_and_close_em_submission_access)

Produces email events when the following occur by calling add_window_email.

  1. Event 151: No Submission Reminder Email (on the 20th of the reporting month)
  2. Event 152: Critical Level 1 Submitted Reminder Email (on the 20th of the reporting month)
  3. Event 155: Open Initial EM Submission Window (at the beginning of the reporting month)
  4. Event 156: Outstanding Emissions Submission Reminder (on the 20th of the reporting month)
  5. Event 157: Emissions Resubmission Window Closed for Non-Submitters (After submission window ends)

add_window_email also stores the following values for use as plugin values in email templates.

Processing Submission Reminder Queue (SubmissionReminderProcessQueue) & Submission Window Queue (SubmissionWindowProcessQueue) Steps

  1. Pull needed emails from EMAIL_TO_PROCESS table where EMAIL_TYPE equals 'submissionReminder' or 'submissionWindow' depending on the method, and STATUS_CD equals 'QUEUED'.
  2. Mark the EMAIL_TO_PROCESS rows' Status Codes as 'WIP' for 'Work-In-Progress'.
  3. Get the list of Fac Ids for the EMAIL_TO_PROCESS rows.
  4. Call CAMD-Services/support/email/emailRecipientList with the list of Fac Ids and an Email Type of EMAIL_TYPE to get the per facility list of (To) recipients for the emails.
  5. Use results from the emailRecipientList endpoint to determine the email addresses for each facility.
  6. For the Fac Id for each EMAIL_TO_PROCESS row produce an EMAIL_TO_SEND row. a. Use Configuration["EASEY_QUARTZ_SCHEDULER_WINDOW_NOTIFICATION_FROM_EMAIL"] to determine the from address which should be the 'ECMPS Submission From Address'. b. Note: Add an environment variables to store the ECMPS Support email address, and include it as a To address. c. Use the EMAIL_TO_PROCESS.EVENT_CD as the TEMPLATE_ID. Need to determine where the templates are stored. d. Set STATUS_CD to 'QUEUED'. e. set CONTEXT to the EMAIL_TO_PROCESS.CONTEXT.

Processing Email Queue Steps (EmailQueue)

  1. Configuration["EASEY_QUARTZ_SCHEDULER_MAX_EMAILS_TO_SEND"] contains the maximum number of in-progress email jobs allowed.
  2. Determine the number of in-progress email jobs by counting the EMAIL_TO_SEND rows with STATUS_CD equals 'WIP'.
  3. Stop processing if the maximum number of in-progress email jobs are running.
  4. Pull the next email to send from the EMAIL_TO_SEND rows where STATUS_CD equals 'QUEUED'.
  5. Mark the EMAIL_TO_SEND row's Status Code as 'WIP' for 'Work-In-Progress'.
  6. Call CAMD-Services/support/email/Process with the EMAIL_TO_PROCESS_ID (likely should be EMAIL_TO_SEND_ID) to send an email.
  7. Return to step 2.

Sending Email (MailTemplateService)

Steps:

  1. Get the EMAIL_TO_SEND row using the passed id.
  2. Get the EMAIL_TEMPLATE row using the EMAIL_TO_SEND.TEMPLATE_ID.
  3. Parse the context from the EMAIL_TO_SEND.CONTEXT.
  4. Get and format the template using the context.
  5. Send the email.
  6. Set the EMAIL_TO_SEND row's Status Code as 'COMPLETE'.
djw4erg commented 4 months ago

Email Text (ECMPS 2.0)

  1. The CAMDECMPSAUX.EMAIL_TEMPLATE table indicates the text template information for each email event.
  2. The EMAIL_TEMPLATE.TEMPLATE_ID contains the ECMPS 1.0 Event Number.
  3. The EMAIL_TEMPLATE.TEMPLATE_SUBJECT contains the subject line for the email.
  4. The EMAIL_TEMPLATE.TEMPLATE_LOCATION indicates the location of the email body template in the [EASEY_CONTENT] repository. Note that the [EASEY_CONTENT] repository contains separate folders for each environment and the location would be the path within each of those folders.
  5. The email body templates may include replacement plugins surrounded by square brackets that the email logic will need to replace.
  6. An Email Template already exists for Event 151 and it should serve as a model for the templates for the other four events mentioned in the ECMPS 1.0 section below.

Email Text (ECMPS 1.0)

_Note that the following includes the plugin [EM_SUB_ACCESS_LIST] which has the following format_:

"{PLANT_NAME}, '{PLANT_STATE} ({ORIS_CODE}) {LOCATION_LIST} - Emissions File for {REPORTING_PERIOD} (as of {EM_SUB_ACCESS_OPEN_DATE})"

Event 151: The "No" Submission Reminder Email

As of the date and time listed below, the EPA has not received your Quarterly Emissions Report(s) for the following facility and monitoring location(s):

[EM_SUB_ACCESS_LIST]

This is a reminder that Quarterly Emissions Reports are due no later than the 30th of the current month.

If you have any questions regarding this correspondence, please contact your EPA Monitoring/Emissions Reporting analyst as soon as possible. For analyst contact information, see CAMD Staff Contacts.

Thank you for your attention to this matter.

Event 152: The Emissions Critical Level 1 Reminder Email

As of the date and time listed below, the following Quarterly Emissions Report(s) were received by the EPA with critical errors:

[EM_SUB_ACCESS_LIST]

This is a reminder that all critical errors must be resolved and the revised Quarterly Emissions Report(s) must be submitted to the EPA within 30 days of the date of the ECMPS feedback letter (see ECMPS feedback letter for details).

If you are unable to meet this deadline, or if you have any questions regarding this correspondence, please contact your EPA Monitoring/Emissions Reporting analyst as soon as possible. For analyst contact information, see CAMD Staff Contacts.

Thank you for your attention to this matter.

Event 155: Open Initial EM Submission Window

This email is to announce the opening of the quarterly emissions submission reporting period for the following:

[EM_SUB_ACCESS_LIST]

This is a reminder that Quarterly Emissions Reports are due no later than the 30th of the current month.

If you have any questions regarding this correspondence, please contact your EPA Monitoring/Emissions Reporting analyst as soon as possible. For analyst contact information, see CAMD Staff Contacts.

Thank you for your attention to this matter.

Event 156: Outstanding Emissions Submission Reminder

As of the date and time listed below, the EPA has not received your Quarterly Emissions Report(s) for the following facility and monitoring location(s):

[EM_SUB_ACCESS_LIST]

This is a reminder that Quarterly Emissions Reports are considered past due, and should be submitted as soon as possible, but no later than 30 days from the date of this email.

If you have any questions regarding this correspondence, please contact your EPA Monitoring/Emissions Reporting analyst as soon as possible. For analyst contact information, see CAMD Staff Contacts.

Thank you for your attention to this matter.

Event 157: Emissions Resubmission Window Closed for Non-Submitters

As of the date and time listed below, the EPA has closed the resubmission window for the following emissions files. No resubmissions were received for these files:

[EM_SUB_ACCESS_LIST]

If you have any questions regarding this correspondence, please send an email to ecmps-support@camdsupport.com as soon as possible.

Thank you for your attention to this matter.

djw4erg commented 4 months ago

CBS Email Recipients API Endpoint Information

The CBS email recipients API endpoint is on NCC dev (https://cbsstagei.epa.gov/CBSD) and staging (https://cbsstagei.epa.gov/CBS). It will be migrated to NCC beta (https://cbsprodbeta.epa.gov/CBS) later this week.

The email recipients endpoint can be accessed by appending “/api/auth-mgmt/emailRecipients” to the URL.

All parameters for this endpoint should be passed in the request body:

emailType (required; possible values are "SUBMISSIONREMINDER", "WINDOWNOTIFICATION") plantIdList (required for SUBMISSIONREMINDER and WINDOWNOTIFICATION; array of FAC_ID values)

Authentication:

SUBMISSIONREMINDER and WINDOWNOTIFICATION Combines bearer token and x-client-id in request header

Sample request bodies:

{

"emailType": "SUBMISSIONREMINDER",

"plantIdList": [1,3,5]

}

Sample response (SUBMISSIONREMINDER):

[

{

    "emailAddressList": "Trey Lightsey <CAMD_BP_3.1_CBS_Rearch_Mail_Test@easternresearchgroup.onmicrosoft.com>",

    "plantIdList": [

        5,

        3,

        1

    ]

},

{

    "emailAddressList": "Brad Vick <CAMD_BP_3.1_CBS_Rearch_Mail_Test@easternresearchgroup.onmicrosoft.com>",

    "plantIdList": [

        3,

        1,

        5

    ]

},

{

    "emailAddressList": "Toya Williams <CAMD_BP_3.1_CBS_Rearch_Mail_Test@easternresearchgroup.onmicrosoft.com>",

    "plantIdList": [

        5,

        1,

        3

    ]

},

{

    "emailAddressList": "James Bice <CAMD_BP_3.1_CBS_Rearch_Mail_Test@easternresearchgroup.onmicrosoft.com>",

    "plantIdList": [

        3,

        1

    ]

},

{

    "emailAddressList": "Submitter Agent-Two <CAMD_BP_3.1_CBS_Rearch_Mail_Test@easternresearchgroup.onmicrosoft.com>",

    "plantIdList": [

        3

    ]

},

{

    "emailAddressList": "Zhiqin Ma <CAMD_BP_3.1_CBS_Rearch_Mail_Test@easternresearchgroup.onmicrosoft.com>",

    "plantIdList": [

        5,

        1,

        3

    ]

}

]

mark-hayward-erg commented 3 months ago

Acceptance Criteria:

SUBMISSIONREMINDER (Old 151 and 152 events)

WINDOWNOTIFICATION (Old 155 and 156 events)