Document Upload Migration Roll out plan

[austinjprice]

Rollout plan document:

https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/teams/vsa/teams/disability-experience/team-docs/Release%20Plans/DBEX%20Document%20Upload%20Service%20Migration%20Release%20Plan.md

Acceptance criteria:

• [ ] Investigate/acquire LH production access for release
• [ ] Go through document and revise to fit this migration
• [ ] Document and answer open questions
• [ ] Document edge cases and failure points
• [ ] Monitoring
• [ ] Test plan and results for known bugs

[austinjprice]

Link to epic containing known EVSS file upload errors: https://app.zenhub.com/workspaces/disability-benefits-experience-team-carbs-6470c8bfffee9809b2634a52/issues/gh/department-of-veterans-affairs/va.gov-team/62913

[NB28VT]

Just going to be posting some notes here for @SamStuckey's benefit on rollout-related things I notice while working on this ticket.

First one I bet Sam has already thought about but just in case:

Quick background:

• SupportingEvidenceAttachment records are retrieved by guid when the SubmitUploads job runs here
• This class inherits from FormAttachment which is an ActiveRecord class and a single-table inheritance model as you probably know.
• Based on the Flipper, the Lighthouse equivalent LighthouseSupportingEvidenceAttachment records will start getting instantiated, retrieved in a job and uploaded instead as we migrate to Lighthouse.

Recall Flipper is used to progressively apply new features to a higher percentage of users.

First off I'm not sure how Flipper works in enqueued jobs which run asynchronous from va.gov API calls from logged in users. Is Flipper smart enough to know the records the job is referencing are tied to a specific user? I don't think so. So I think we will have to figure out how Flipper gates will work in jobs first. How does the job decide on the state of the Flipper if it doesn't know about a specific User?

Regardless, this will raise the issue of SupportingEvidenceAttachment records being created for users when they have the Flipper disabled, and then when they finally submit which we have no control over recall, they may take months to fill out the form, it's possible the Flipper switch will have flipped, and we will attempt to find a LighthouseSupportingEvidenceAttachment record, which won't exist because the guid refers to a FormAttachment record of the SupportingEvidenceAttachment subclass.

This is assuming SupportingEvidenceAttachment records get created whenever a user uploads a document, not when we bundle everything together to upload when they submit. I think that's the case

We can probably account for this with something like the below:

module EVSS
  module DisabilityCompensationForm
    class SubmitUploads < Job
    ...

    with_tracking("Form526 Upload: #{guid}", submission.saved_claim_id, submission.id) do
      if Flipper.enabled?(:disability_compensation_lighthouse_document_service_provider)
        # Note fallback
        sea = LighthouseSupportingEvidenceAttachment.find_by(guid:) || SupportingEvidenceAttachment.find_by(guid:)
        ....
      else
        sea = SupportingEvidenceAttachment.find_by(guid:)
        ....
      end

      ......
    end
  end

But I'm not sure what cascading complications this causes, and whether it's worth considering running a one-off job that updates the subtypes of SupportingEvidenceAttachment records to LighthouseSupportingEvidenceAttachment for a subset of users before we flick the switch for them? I think we have control over which exact users have a switch flipped.

Does that make sense? May be easier to discuss on a call but an interesting problem to consider here.

[SamStuckey]

Current work for the rollout plan is to move through this document and update it for our use case. This is just a copy paste of a similar DBEX rollout plan https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/teams/vsa/teams/disability-experience/team-docs/Release%20Plans/DBEX%20Document%20Upload%20Service%20Migration%20Release%20Plan.md

[SamStuckey]

@NB28VT " First off I'm not sure how Flipper works in enqueued jobs which run asynchronous from va.gov API calls from logged in users. Is Flipper smart enough to know the records the job is referencing are tied to a specific user? I don't think so. So I think we will have to figure out how Flipper gates will work in jobs first. How does the job decide on the state of the Flipper if it doesn't know about a specific User? " ^^ I know this is already on someones todo list. we should check with team one before we plan on making any mods here.

[NB28VT]

Noting for refinement I think the original acceptance criteria for this still applies; I've already started brainstorming the answers we will need to get from team 1, Lighthouse, VBMS etc. but it's still a little early to scope the effort involved in getting testing set up e2e

[NB28VT]

Closing as this has been split up into smaller tickets now that I have a better handle on what's involved. The first one is https://github.com/department-of-veterans-affairs/va.gov-team/issues/70109

[NB28VT]

Closing this as we will be tracking the release and rollout for each job update separately