A Data release is composed of a set of data that has been verified to be releasable; this includes both Clinical and Genomic.
Need a snapshot of data files and clinical data at the point of Data Release.
From the snapshot, we need to generate the clinical export. Read from the snapshot and generate it into the clinical export TSVS. Assemble based on a Donor selection from the UI.
Tasks:
[ ] Define the index & snapshot structure
--- Needs to include clinical data.
--- Needs to include file-tags to be added.
[ ] Complete the swap over of the File-Service application for maestro.
-- make sure file data is being correctly indexed.
-- all files are currently in state RELEASED
[ ] Extend Files-Service to reach out to clinical and include clinical data into the new index.
--- program-x-re_1 index times n programs as a file repository index.
--- on a daily bases, the ETL manager will reindex
------ files form the gateway that are NOT released for donors that are NOT YET EVER released
------ get the clinical data for those donors
------ build program-x-unreleased_1
--- Each program has a released index, and an unreleased index . The unleased index is
[ ] To download clinical data, we need to do that as gateway-api code. It will be a zipped file export. This is likely supported by another application. Gateway could fetch and compile the TSVs.
How does it know the structure of the TSVs from the zip file? <-- to be answered.
[ ] Storing the data does not necessarily need to be the search index format; it can be a snapshot with the data dictionary version
The Outcome of this Epic is to
[ ] Be able to download all released clinical data for a project from the File Repository
[ ] The format of the files can mimic the format used to submit data to clinical.
[ ] Be able to search by some clinical fields (like primary site) in the facet panel, display that in the table and create a new graph for primary site.
For the set of Dono
verified
to be releasable; this includes both Clinical and Genomic.Tasks:
[ ] Define the index & snapshot structure --- Needs to include clinical data. --- Needs to include
file-tags
to be added.[ ] Complete the swap over of the
File-Service
application for maestro.-- make sure file data is being correctly indexed. -- all files are currently in state
RELEASED
[ ] Extend
Files-Service
to reach out to clinical and include clinical data into the new index. ---program-x-re_1
index times n programs as afile repository
index. --- on a daily bases, the ETL manager will reindex ------ files form the gateway that are NOT released for donors that are NOT YET EVER released ------ get the clinical data for those donors ------ buildprogram-x-unreleased_1
--- Each program has a released index, and an unreleased index . The unleased index is[ ] To download clinical data, we need to do that as gateway-api code. It will be a zipped file export. This is likely supported by another application. Gateway could fetch and compile the TSVs. How does it know the structure of the TSVs from the zip file? <-- to be answered.
[ ] Storing the data does not necessarily need to be the search index format; it can be a snapshot with the data dictionary version
The Outcome of this Epic is to