icgc-argo / argo-clinical

Clinical data submission for ARGO programs.
GNU Affero General Public License v3.0
2 stars 0 forks source link

Core Complete Calculation logic using Missing Entity Exception #1111

Closed joneubank closed 5 months ago

joneubank commented 5 months ago

Link to Issue

Description

Core completion calculation needs to take into consideration whether a donor has a MissingEntityException. If they do, then the Treatment and Follow-up entities are not required to be core complete.

The mechanism for calculating core completion stats has been standardized - there are no longer different calculation cases that behave differently based on overrides or completion state. Now, core completion calculation will always result in the same result for a given donor.

Core-Completion is triggered by:

  1. Data registration
  2. Data submission
  3. Dictionary migration

This removes the patch endpoint that allows admin to "override" core calculation stats. The CompletionStats object has had this override property removed and all the calculation logic that used these overrides are removed. A migration is included to remove all overrides from teh database. It has been checked that this feature was never used in the current ARGO data set so this migration is added to ensure consistency, though we don't expect to have any actual cases to update.

A caching utility for MissingEntityExceptions was created as a performance consideration to help when fetching exceptions for many donors. Using this cache to access the MissingEntityException repo will prevent repeatedly fetching exceptions for the same program when you are fetching exceptions for a long list of donors.

Checklist

Type of Change

Checklist before requesting review: