Path to Production

[halprin]

DevEx/OpEx

We need one location to document the items we need to complete before we're ready for real real production. The below items apply both to the Intermediary and the RS SFTP Ingestion services. Feel free to make separate backlog items for these tasks if they are warranted. Some may already exist.

README

All unchecked tasks have been spun into their own tickets (see comment below).

Tasks

• [ ] Latest version deployed to production environment.
• [x] Monitoring dashboards created.
• [x] Monitoring alerts created to send to our Slack channel.
• [x] Log alerts created to send to our Slack channel.
• [x] PagerDuty (or equivalent) set-up. All engineers are in the rotation.
• [x] Monitoring or log alerts created to send to PagerDuty.
• [x] All critical and high findings in Fortify are fixed.
• [x] All critical and high findings in Snyk are fixed.
• [ ] Review SonarCloud findings and see fix anything deemed necessary.
• [x] Gameday exercises are ran.
• [ ] Re-review our incident management process. Correct or update anything.
• [ ] Disaster recovery plan exists and is documented (data back-ups, etc.).
• [ ] Super e2e tests exist for testing important initial implementation's usecases.
• [x] Both threat models (TI and ingestion service) are up-to-date.
• [ ] Have an elasticity or scale-out plan. This doesn't need to be a document, but all the services used should have some explanation as to how things will scale.
• [x] Production secrets have been set
• [x] ReportStream config is in production (at least ca-phl, not sure of others) - we may need to get RS to do this step for us
• [ ] CRON expression is updated in prod to turn on polling for CADPH
• [ ] Create a rollback plan for when a release goes awry.

Additional Context

Add any other context or screenshots about the work here.

[JohnNKing]

Opportunities to refine this? (e.g. break up into smaller chunks?)

[halprin]

@JohnNKing, this is more of a reminder issue than an actual story where everything needs to be done inside.

[scleary1cs]

Retention Policy Note (from 11/01/24 toilet session): Company XYZ Data Retention Policy

1. Purpose To define how Company XYZ manages data retention, ensuring compliance with regulatory requirements and supporting business operations.
2. Scope This policy applies to all data collected, processed, and stored by Company XYZ, including employee, customer, and business data.
3. Retention Periods by Data Type Customer Data (e.g., names, contact info, purchase history) Retain for 7 years after the last interaction. Justification: Business needs, customer service, and legal requirements. Employee Records (e.g., employment history, performance reviews) Retain for 10 years after employment ends. Justification: Legal requirements and future employment verification. Financial Records (e.g., invoices, expense reports) Retain for 7 years to comply with tax regulations. Marketing Data (e.g., prospect lists, marketing analytics) Retain for 3 years from collection date. Justification: Ensures data relevance while supporting future campaigns.
4. Data Deletion Procedures Upon reaching the end of the retention period, data is securely deleted or anonymized using industry-standard methods. Automated deletion processes should be in place where possible, with periodic audits to confirm compliance.
5. Exceptions Data required for ongoing litigation or audit may be retained beyond standard periods. Requests for exceptions must be submitted in writing to the Compliance Officer.
6. Roles and Responsibilities Data Protection Officer: Ensures compliance with data privacy laws. IT Department: Implements technical aspects of data retention and deletion. Compliance Team: Audits data retention practices annually.
7. Policy Review and Updates This policy is reviewed annually and updated as needed to reflect regulatory changes or business needs.
8. Enforcement and Penalties Non-compliance with this policy may result in disciplinary action, up to and including termination.
9. Legal Compliance This policy aligns with relevant regulations, such as GDPR, CCPA, and industry-specific data protection standards. Key Takeaways:

Clarity and Specificity: Lists specific retention periods based on data types. Regular Audits: Ensures continuous compliance. Automated Processes: Reduces human error and administrative overhead. Legal Alignment: Ensures the policy meets regulatory requirements. This structure helps in creating a clear, enforceable, and compliant data retention policy.

[halprin]

Noes from earlier today on what should be their own cards and priority.

• Path to Production
  • Before Go Live.
    • Rollback plan.
    • SonarCloud findings.
    • Review incident management plan.
      • Can be done with Claire and SMEs.
  • Before end of year.
    • Disaster recovery plan. Make sure if we delete data in DB, we can recover that data. Actually exercise this. Other things too like our Docker instances.
    • Super e2e tests from Daniel and Kathy.
  • Others
    • Production Smoke Tests.
    • Data retention policy is documented.

[scleary1cs]

Rollback plan - Rollback plan #1530 SonarCloud findings - SonarCloud findings #1531 Review incident management plan (Can be done with Claire and SMEs) - Review incident management plan #1534

Before end of year. Disaster recovery plan. Make sure if we delete data in DB, we can recover that data. Actually exercise this. Other things too like our Docker instances. - Disaster recovery plan #1537 Super e2e tests from Daniel and Kathy. - Add SME's CA test case scenarios to RS e2e #1539

Others Production Smoke Tests - Production Smoke Tests #1541 Data retention policy is documented - Data retention policy #1542