[Research Spike] Mechanism for OFA users to vet data submissions made after deadlines

[lfrohlich]

Description: Original user story: As an OFA analyst I want to be able to control what is added to the database Also pulled in original ticket #1972 [Research Facilitation] Workshop policy/decisions on the number of fiscal years to surface in TDP Decide the number of fiscal years needed to be available/made resubmittable within TDP

For example, do we want to impose deadlines/windows for submission and resubmission? Initially--any data files submitted after a deadline will go into "purgatory" where only OFA Admin or Sys Admin can trigger the data validation and db storage process. STTs will be notified of this as well since its different from the typical data submission process we're building toward.

Open Questions: when this deadline should be. Based on historical patterns, this deadline is typically 2 years after the first quarter of data for the fiscal year is due. This is to allow time for resubmissions covering the full fiscal year in time for OFA's annual data publications. once we decide on how to calculate the deadline (manual or date trigger), we need dev to scope out the tasks for implementation.

Do grantees need some in-between state of submitted late / outside of acceptance/rejection/feedback process

    ○ There are standard cases for how long we should retain data and then exceptional cases like "We discovered a massive problem with data from 2015 that needs to be fixed" 
        ▪ Flip-ability of a switch to allow submission for edge cases like this?
        ▪ Currently somebody could submit something to the wrong quarter and even if it errors we would have the file.
        ▪ Also possible we could invest in some generic edge case file upload bucket (enabled for specific STTs on demand?) "Open a late data submission ticket" kind of flow?
        ▪ Does this relate to the ask from some folks we've heard of "can we test our files without actually submitting?" -- We have a few folks doing this via staging
    ○ Beyond 4.0 priority
    ○ Look into whether we have additional related tickets already for some of this -- Data retention policy
        ▪ #1288 Updated SORN with backup retention policy

2267 As a grantee, I want to see 5 years of data submission history

1288

2267

Tasks:

• [ ] Sync with DIGIT to understand current state data submission deadline requirements
• [ ] Make a decision with DIGIT as to what new deadlines TDP should enforce
• [ ] Create follow-on design tickets to accommodate design notifications to alert DIGIT

[amilash]

@lfrohlich Lets talk about the purpose of this ticket. I think we need to rewrite it. Is the purpose here to try and have a way to systemically prevent users from uploading and overwriting the wrong years or time periods during submission?

If thats the case, have we mitigated that by prompting the user to select the year and quarter for submission? Or are we tying prevent or to do something else here?

[ADPennington]

adding context here:

Currently, with TDRS, STTs can continuously submit data for any fiscal period, even after data deadlines. When this happens, data in the database are overwritten and there is no db back-up. If resubmitted data have changed in any way since OFA's production of data reports, it is not possible to reproduce those numbers (unless we download extracts from the db). Relatedly, this has raised questions for us about the recency of the data across STTs and its implications on how we use the data for other purposes (e.g. longitudinal research). This ticket was created in response to these concerns.

I can think of at least a couple of ways this could be addressed:

• facilitate the ability for OFA data users to extract data from db tables by transmission date (e.g. all section 1 data submitted for FY2020Q1 as of 7/13/2021)
• dynamically change the years available in the fiscal year dropdown based on a pre-set cutoff date (e.g. after 7/13/2021, dropdown menu should not include 2020). I worry some about this approach, because there are usually a few reasonable exceptions why STTs submit data very late.
• We can also display a warning message on the frontend when a user is about to send a late submission, which could prompt them to confirm the intention to do this and explain what this late submission could mean.

cc: @lfrohlich @amilash

[lfrohlich]

I think this is something that could benefit from UX research to figure out the best way to do it. I know other systems grantees use lock them out after a certain date and then they need to request access to resubmit. I don't think we want to do that, but it would help to better understand from OFA, including RO, perspective of why deadlines and control of database matters, and from STT perspective why they may need to resubmit. cc: @dk-ui @reitermb @amilash @ADPennington

[dk-ui]

I agree! Ill put this in our next research plan workshop to discuss further.

[ADPennington]

I think we decided as part of the resubmission epic that--at least initially--any data files submitted after a deadline will go into "purgatory" where only OFA Admin or Sys Admin can trigger the data validation and db storage process. STTs will be notified of this as well since its different from the typical data submission process we're building toward. Is this right?

If so, then I think we just need to decide the following:

• when this deadline should be. Based on historical patterns, this deadline is typically 2 years after the first quarter of data for the fiscal year is due. This is to allow time for resubmissions covering the full fiscal year in time for OFA's annual data publications.
• once we decide on how to calculate the deadline (manual or date trigger), we need dev to scope out the tasks for implementation.

If the goal here is to get to parity with TDRS, then this is not something that needs to be implemented until STTs have at least submitted a full fiscal year's worth of data. So r4+ in terms of priority.

cc: @lfrohlich @valcollignon @reitermb @sreedevip

[reitermb]

I think we decided as part of the resubmission epic that--at least initially--any data files submitted after a deadline will go into "purgatory" where only OFA Admin or Sys Admin can trigger the data validation and db storage process. STTs will be notified of this as well since its different from the typical data submission process we're building toward. Is this right?

If so, then I think we just need to decide the following:

• when this deadline should be. Based on historical patterns, this deadline is typically 2 years after the first quarter of data for the fiscal year is due. This is to allow time for resubmissions covering the full fiscal year in time for OFA's annual data publications.
• once we decide on how to calculate the deadline (manual or date trigger), we need dev to scope out the tasks for implementation.

If the goal here is to get to parity with TDRS, then this is not something that needs to be implemented until STTs have at least submitted a full fiscal year's worth of data. So r4+ in terms of priority.

cc: @lfrohlich @valcollignon @reitermb @sreedevip

Quick follow-up question:

If there are no technical/status clarity issues with doing so, is it the data validation process we want to prevent or just acceptance? i.e., Prevent a file from getting automated error/quality feedback or prevent a file from having any of it's data accepted into the database?

[ADPennington]

I think we decided as part of the resubmission epic that--at least initially--any data files submitted after a deadline will go into "purgatory" where only OFA Admin or Sys Admin can trigger the data validation and db storage process. STTs will be notified of this as well since its different from the typical data submission process we're building toward. Is this right? If so, then I think we just need to decide the following:

• when this deadline should be. Based on historical patterns, this deadline is typically 2 years after the first quarter of data for the fiscal year is due. This is to allow time for resubmissions covering the full fiscal year in time for OFA's annual data publications.
• once we decide on how to calculate the deadline (manual or date trigger), we need dev to scope out the tasks for implementation.

If the goal here is to get to parity with TDRS, then this is not something that needs to be implemented until STTs have at least submitted a full fiscal year's worth of data. So r4+ in terms of priority. cc: @lfrohlich @valcollignon @reitermb @sreedevip

Quick follow-up question:

If there are no technical/status clarity issues with doing so, is it the data validation process we want to prevent or just acceptance? i.e., Prevent a file from getting automated error/quality feedback or prevent a file from having any of it's data accepted into the database?

mostly the latter (very late data getting into the db without prior approval), but i dont see much value in even triggering the parsing and validation process until OFA decides we want the data. I think it could also cause some confusion (e.g. if STTs are getting feedback on files that are in purgatory, this might prematurely suggest that we've decided the file is eligible for acceptance). i presume these files would sit in a separate S3 until we've approved. @reitermb

[lfrohlich]

^ makes sense. Moved to v4 and added some of your notes to the description, @ADPennington

[valcollignon]

• [ ] Sync with DIGIT to understand current state data submission deadline requirements
• [ ] Make a decision with DIGIT as to what new deadlines TDP should enforce
• [ ] Create follow-on design tickets to accommodate design notifications to alert DIGIT

Made edits to description, including the tasks above. Added to #1396 epic. Per UX ticket refinement session 12.9.21.

[lfrohlich]

This question just came up with a state wanting to resubmit data found to be erroneous in an audit. Planning to resubmit for FY21 but could resubmit prior years. (MA audit A-01-22-69813, Finding 2021-020)

[robgendron]

@ADPennington please review this ticket.