Beta Bug - Inconsistent Eval and Submission Emails

[ibarra-michelle]

In the Beta environment, the evaluation report and/or submission email is not consistently sent to the user, even though the evaluation and/or submission was completed.

This issue was identified on 09/25/2024 between 11:35am - 12:00pm EST for the following configurations:

1. Comanche, ORIS 470, Loc 1 For the MP eval and submission, I successfully evaluated the MP data and never received an Eval email. I successfully submitted the MP data into CDX (available in submission history), however, I did not receive a submission email.
2. Silverhawk, ORIS 55841, Loc A01 For the EM eval (no submission), I successfully evaluated and received an Evaluation email.

Update: Submission emails are being worked on in #6183. For this ticket we need to investigate the evaluation emails and make whatever changes are necessary so that every evaluation results in an email

Acceptance Criteria

• Every evaluation results in an evaluation email.

[alangmaid]

The submission email logic will be tested as part of #6183. The Evaluation email issue will need to be investigated.

[ibarra-michelle]

The submission email logic will be tested as part of #6183. The Evaluation email issue will need to be investigated.

@alangmaid, should we consider moving this ticket #6409 back to Sprint 14 since the acceptance testing for ticket #6080 Submission Confirmation Emails and Feedback Reports (UAT ticket in Sprint 9), may address this issue?

FYA - @bryanramirez1 , @maheese , @j-tafoya

[yonatan-dp]

Changes Implemented to Fulfill the Acceptance Criteria

To address the issue of missing or inconsistent evaluation emails, I made the following changes to ensure that every evaluation— especially unsuccessful—results in an email notification to the user and ECMPS support. Below is a summary of the modifications:

---

1. Consistent Notification Logic

• I modeled the email notification mechanism for evaluation errors after the submission queueing logic implemented in easey-ui/issues/6391. This keeps evaluations and submissions error notifications consistent.
  • All-or-Nothing Queueing: If any part of the evaluation set cannot be queued, the entire evaluation set is failed, triggering an error email to both the user and ECMPS support.
  • Evaluation Set Tracking: Even if certain evaluation sets fail to queue and do not persist in the database, the generated evaluation set IDs will appear in the email notifications. This enables ECMPS support to correlate these IDs with entries in debug logs (traceability).

---

2. Queueing and Evaluation Processing Errors

• Queueing Errors:
  • Queueing errors are handled within camd-services. Any issues that occur during the initial queueing of evaluation sets trigger an email notification.
• Evaluation Processing Errors:
  • A new API endpoint within camd-services is now available to handle errors that occur during evaluation processing. This endpoint can be invoked by the quartz-scheduler (which manages the evaluation process) to send notification emails for any processing errors.
  • Sample Request to send an evaluation error email

POST {{baseCamdServicesUrl}}/camd-services/email/eval-error

{
    "evaluationSetId": "b2b5f26a-5398-42d7-81b9-b61e57eb4761",
    "evaluationId": 456,
    "rootError": "Error details for testing purposes.",
    "evaluationStages": [
        {
            "action": "EVALUATION_STARTED",
            "dateTime": "2024-10-31T10:00:00"
        },
        {
            "action": "EVALUATION_COMPLETED",
            "dateTime": "2024-10-31T10:05:00"
        }
    ]
}

---

3. Example Screenshots

Example Email to ECMPS Support

Example Email to User

---