[DMC Updates] Statistician New Data Extract for Organs

ouhft / COPE

Project Repository for Work Package 4 of the COPE Transplant Trial

https://cope.nds.ox.ac.uk

1 stars 0 forks source link

[DMC Updates] Statistician New Data Extract for Organs #314

Closed marshalc closed 6 years ago

marshalc commented 6 years ago

From #309 : CM will flatten the data into a single spreadsheet for LD to analyse. This spreadsheet will have YYY rows - one for each kidney randomised - and include (where applicable/available)

Trial ID
Included/excluded and reason for exclusion
All donor demographics
All procurement detail
All recipient demographics
All transplant procedure data
All follow-up data for each timepoint
Total number of AEs/SAEs for that kidney
Recipient death and date of death (for included kidneys) - defined by death reported on an SAE form
Permanent graft failure (for included kidneys) - defined as report of permanent failure on an SAE form, or graft failure/graft removal on a follow-up form
Delayed graft function (for included kidneys) - defined as dialysis on days 1-7 post-transplant or selection of a reason for dialysis (but not if graft reported as removed on initial follow-up form)

marshalc commented 6 years ago

So far, data extract for all Organs working with Organ, Donor, Allocation, Recipient, Followup records in place. Still to add the S/AE details.

marshalc commented 6 years ago

Querying the need for the DGF column with SK. Have attempted to list any S/AE with a Category of PGF for the value in the PGF column.

Concerned that the processing time for this data extract is likely to exceed the server process timeout when it goes live...

marshalc commented 6 years ago

Confirmed that PGF data can only be listed based on the categorisations done by the central investigator, so that is working as expected. Looking at whether the DGF calculation now will increase the processing time too much to include (confirmed it's not essential, but is desirable).

marshalc commented 6 years ago

Processing time is currently taking 80-90 seconds for the data extract, on the laptop, which means it will take longer on the server and likely fail with a timeout. Trying to see if there's anything left I can do to speed this up...

marshalc commented 6 years ago

...and a sprinkling of extra select_related and prefetch_related calls, and we're now looking at 40-45 seconds with the extra data columns included. Going to see if I can merge this into the master branch now, and do a partial 0.9.0 release so that I can test the data extract and notify LD if it is working.

marshalc commented 6 years ago

Data extract took under 10 seconds to run on Production, confounding typical processing performance differences!

Extract is now available on production after a partial release of the 0.9.0 package

marshalc commented 6 years ago

Included/excluded and reason for exclusion -- This still needs to be manually curated, with the potential for an initial setting to be defined by some as yet undetermined rules.