VEuPathDB / EdaNewIssues

0 stars 0 forks source link

VDI: Allow release of studies without integrating data into EDA #665

Closed danicahelb closed 1 month ago

danicahelb commented 1 year ago

Since we do not have bandwidth to integrate all datasets we have in hand, we want to have the option to support downloads of flat data files (ie, the files just as given to us from the provider).

For these studies that we won't be integrating, we want to:

This is important as it shows potential funders that many groups are interested in working with ClinEpiDB and that we already have the data in hand and are ready to integrate these studies when funding is made available.

dmfalke commented 1 year ago

I think we might want something that looks like this: image

These are the pieces we would likely need:

  1. WDK dataset record (check with JB).
  2. Download service capable of serving files for studies without EDA data loaded (check with Ryan).
  3. Updates to the front end to render a page as above. This might require a new attribute in the dataset record, or we might be able to do it based on the absence of an EDA study.

I can't speak for 1 and 2, but I think 3 would take under a week to implement.

jaycolin commented 1 year ago

Updates to the front end to render a page as above. This might require a new attribute in the dataset record, or we might be able to do it based on the absence of an EDA study.

I don't think we do without the EDA study record, because the tuning manager needs it.

Can WDK render the Download tab based on whether the download path exists, for versions that exist? ex. for GEMS1A above, on fir: /var/www/Common/apiSiteFilesMirror/downloadSite/ClinEpiDB/release-25/2a6ace17a19ade4372f33c71cb2a591861d1915c/ ..same for releases 21,14,6

dmfalke commented 1 year ago

I don't think we do without the EDA study record, because the tuning manager needs it.

I wonder if part of the solution here is to load an "empty" study record. By "empty", I mean a study with no entities or variables. When the front end encounters this, then we render something like the screenshot in that issue.

Can WDK render the Download tab based on whether the download path exists, for versions that exist? ex. for GEMS1A above, on fir: /var/www/Common/apiSiteFilesMirror/downloadSite/ClinEpiDB/release-25/2a6ace17a19ade4372f33c71cb2a591861d1915c/ ..same for releases 21,14,6

I don't know how to answer this question :slightly_smiling_face:. Maybe my response above obviates this?

jaycolin commented 1 year ago

I wonder if part of the solution here is to load an "empty" study record. By "empty", I mean a study with no entities or variables. When the front end encounters this, then we render something like the screenshot in that issue.

Yes that is how we currently load it, ex. the Gates REACH LAKANA study: https://qa.restricted.clinepidb.org/ce.restricted.qa/app/workspace/analyses/DS_515a92c711 This would be a valid download link (this file exists), if it had the AuthKey param: https://qa.restricted.clinepidb.org/eda/download/ClinEpiDB/PREVIEW_Gates_REACH_LAKANA_rct/release-29/test.txt I have only removed ?Auth-Key=****** which would be part of the file URL in a normal Download tab

Would this work, easy, not easy?

dmfalke commented 1 year ago

@jaycolin thanks for the info! The eda service returns a 500 when requesting that study data:

{"status":"server-error","message":"Found no entities for study: PREVIEW_Gates_REACH_LAKANA_rct","requestId":"xxx"}

The current behavior of the front-end is to redirect to a stand-alone study page. We can change that to render a download tab with the flat files. @danicahelb would this behavior be okay?

danicahelb commented 1 year ago

Confirmed with @dmfalke

  1. If prerelease, go to standalone page

  2. if eda study has no data loaded and downloads exist, always show download tab (do NOT show B&S & Viz tabs). if study has controlled or protected access restrictions, clicking on the links to file downloads brings the user to the access request form (unless access had previously been granted)

danicahelb commented 1 year ago

For Namibia (the 1st study that will be released with no data integrated into the EDA), I have added a new study to the presenter:

For the version with no data but downloadable provider files: datasetPresenter name="ISASimple_Gates_Namibia_RACD_rfMDA_RAVC_download_RSRC" <prop name="studyAccess">private</prop> <prop name="isPublic">false</prop>

aurreco-uga commented 1 month ago

this got released in branch api-build-68-b