10-10CG - Zero Silent Error checklist

[hdjustice]

Benefit to the Veteran

As a Veteran, I want to be notified in the event that a failure occurs with my 10-10CG application, so that I may address it and make any adjustments to ensure a successful submission.

Background

VA Platform initiative

• This document details how your Veteran-facing team can perform discovery tasks to determine if your application (product, service, et al.) has a silent failure.

Thank you for helping us find and fix silent failures. It is our collectively responsibility as stewards of the Veteran's data to track these problems down and fix them. We appreciate your support and dedication in helping us reach the goal of zero silent errors. This is a blameless process. We want to thoroughly investigate this problem so that we know the depth and breadth of the problem and can take the necessary steps to address it.

What is and is not a silent failure? (expand for details)

- What is a silent failure? - Silent failures are any errors that occur within your application on any public facing VA platform (VA.gov, mobile application, VA Notify, et. al.) whereby the user (Veteran, claimant, 3rd party, etc.) interacting with your application is NOT made aware that an error has occurred. - What is NOT a silent failure? - If an application returns an error to the user immediately upon detection of the error, then the error is not silent. - If an application sends an email to the user notifying them that an error with their transaction has occurred, then the error is not silent (so long as sending the email is also monitored and confirmed to have been sent).

Scope

• Determine whether the 10-10CG has any silent errors that occur, document them (if any) and create a plan to address them quickly.

Tasks

• [x] Review the 10-10CG checklist as a team
• [ ] Determine if we have any silent failures on the 10-10CG
• [ ] Document the silent failures
• [ ] Create tickets for investigation, monitor setup and documentation, if any is needed on addressing silent failures
• [ ] Inform PO of results, and remediation plan if needed.
• [ ] If no silent failures are found, notify the #zero-silent-failures Slack channel (mention Team (10-10 Health Apps) & Portfolio (Health Tools), and that no silent failures were found)

Acceptance Criteria

• The team has gone through the checklist for the 10-10CG and found no silent failures present.

[hdjustice]

10-10CG zero silent failures checklist to-do items:

• Using APIs?
  • Are we using EVSS Document Upload API?
• Did your application use the same APIs when it shipped as it does today?
  • need to do research on this, I do know that we did stop using EVSS APIs, as well as moved from CARMA directly to MuleSoft. This would be a Yes, and we need to determine what they were and what they are now for any remediation
• Datadog Monitoring tagging needed?
  • Luis is aware of this
• Have you filed issues for errors that are appearing in Datadog / Slack?
  • Need clarification on this question, we open team tickets, but we do not file them with categories
• Understanding how errors are handled when each system in the path fails, in maintenance or completely down?
  • Check overview doc, may need to create diagram
• System receiving data indicating how fatal errors are handled?
  • Need this outlined in writing from Mulesoft/CARMA
• Capture and notify user of all potential points of failure?
  • Need to check CARMA (noted above)

[hdjustice]

Converted to Epic and stories created to address outstanding checklist questions.