search

pharmaverse / admiral

ADaM in R Asset Library
https://pharmaverse.github.io/admiral
Apache License 2.0
224 stars 64 forks source link

derive_var_obs_number creating unexpected changes to TRTA and TRTP (ADVS) #1357

Closed morrishj closed 2 years ago

morrishj commented 2 years ago

What happened?

TRTA and TRTP are derived correctly during the first half of the ADVS template, but when I get to the following section of code, both variables are changed and start showing large numbers of NA's. 

advs <- advs %>%   # Calculate ASEQ   derive_var_obs_number(     new_var = ASEQ,     by_vars = vars(STUDYID, USUBJID),     order = vars(PARAMCD, ADT, AVISITN, VISITNUM, VSSEQ),     check_type = "error"   )

Essentially, before I run that code, they are correct, and after I run that code, they are wrong.

As far as I can tell, that code shouldn't have any impact on TRTA and TRTP. TRT01A and TRT01P still stay the same.

I have cleared the workspace and restarted the session, and the same thing happens each time.

Session Information

sessionInfo() R version 4.1.3 (2022-03-10) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 20.04.4 LTS

Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8
[6] LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] stringr_1.4.0 lubridate_1.8.0 admiral_0.7.0 dplyr_1.0.9

loaded via a namespace (and not attached): [1] rstudioapi_0.13 magrittr_2.0.3 hms_1.1.1 getPass_0.2-2 tidyselect_1.1.2 R6_2.5.1 rlang_1.0.3 fansi_1.0.3 httr_1.4.3
[10] tools_4.1.3 utf8_1.2.2 DBI_1.1.3 cli_3.3.0 ellipsis_0.3.2 assertthat_0.2.1 tibble_3.1.7 lifecycle_1.0.1 crayon_1.5.1
[19] tidyr_1.2.0 purrr_0.3.4 readr_2.1.2 tzdb_0.3.0 vctrs_0.4.1 curl_4.3.2 rice_3.0 glue_1.6.2 haven_2.5.0
[28] stringi_1.7.6 compiler_4.1.3 pillar_1.7.0 forcats_0.5.1 generics_0.1.2 jsonlite_1.8.0 pkgconfig_2.0.3

Reproducible Example

vs <- convert_blanks_to_na(vs)

---- Lookup tables ----

Assign PARAMCD, PARAM, and PARAMN

param_lookup <- tibble::tribble( ~VSTESTCD, ~PARAMCD, ~PARAM, ~PARAMN, "SYSBP", "SYSBP", "Systolic Blood Pressure (mmHg)", 1, "DIABP", "DIABP", "Diastolic Blood Pressure (mmHg)", 2, "PULSE", "PUL", "Pulse Rate (beats/min)", 3, "WEIGHT", "WGHT", "Weight (kg)", 4, "HEIGHT", "HGHT", "Height (cm)", 5, "TEMP", "TEMP", "Temperature (C)", 6, "RESP", "RESP", "Respiratory Rate (breaths/min)", 7, "OXYSAT", "OXYSAT", "Oxygen Saturation (%)", 8 ) attr(param_lookup$VSTESTCD, "label") <- "Vital Signs Test Short Name"

Assign ANRLO/HI, A1LO/HI

range_lookup <- tibble::tribble( ~PARAMCD, ~ANRLO, ~ANRHI, ~A1LO, ~A1HI, "SYSBP", 90, 130, 70, 140, "DIABP", 60, 80, 40, 90, "PUL", 60, 100, 40, 110, "TEMP", 36.5, 37.5, 35, 38, "RESP", 8, 20, NA, NA )

ASSIGN AVALCAT1

avalcat_lookup <- tibble::tribble( ~PARAMCD, ~AVALCA1N, ~AVALCAT1, "HGHT", 1, ">100 cm", "HGHT", 2, "<= 100 cm" )

---- User defined functions ----

Here are some examples of how you can create your own functions that

operates on vectors, which can be used in mutate().

format_avalcat1n <- function(param, aval) { case_when( param == "HGHT" & aval > 140 ~ 1, param == "HGHT" & aval <= 140 ~ 2 ) }

---- Derivations ----

Get list of ADSL vars required for derivations

adsl_vars <- vars(TRTSDT, TRTEDT, TRT01A, TRT01P, AP01SDTM)

advs <- vs %>%

Join ADSL with VS (need TRTSDT for ADY derivation)

derive_vars_merged( dataset_add = adsl, new_vars = adsl_vars, by_vars = vars(STUDYID, USUBJID) ) %>%

Calculate ADT, ADY

derive_vars_dt( new_vars_prefix = "A", dtc = VSDTC, flag_imputation = "none" ) %>%

calculate astdy and aendy

derive_vars_dy(reference_date = TRTSDT, source_vars = vars(ADT)) %>%

calculate adtm

derive_vars_dtm(new_vars_prefix = "A", dtc=VSDTC, flag_imputation="both")

advs <- advs %>%

Add PARAMCD only - add PARAM etc later

derive_vars_merged( dataset_add = param_lookup, new_vars = vars(PARAMCD), by_vars = vars(VSTESTCD) ) %>%

Calculate AVAL and AVALC

mutate( AVAL = VSSTRESN, AVALC = VSSTRESC, AVALU = VSSTRESU )

get visit info

advs <- advs %>%

Derive Timing

mutate( AVISIT = ifelse(!is.na(ADTM) & ADTM<= AP01SDTM, "BASELINE", VISIT) )

advs <- advs %>%

Derive Timing

mutate( AVISIT = ifelse(str_detect(VISIT,"(?i)Unscheduled"), "Unscheduled", AVISIT) )

advs <- advs %>%

Derive Timing

mutate( AVISITN = ifelse(AVISIT == "BASELINE", 0, VISITNUM) )

advs <- advs %>%

Calculate ONTRTFL

derive_var_ontrtfl( start_date = ADT, ref_start_date = TRTSDT, ref_end_date = TRTEDT, filter_pre_timepoint = AVISIT == "Baseline" )

Calculate ANRIND : requires the reference ranges ANRLO, ANRHI

Also accommodates the ranges A1LO, A1HI

advs <- advs %>% derive_vars_merged(dataset_add = range_lookup, by_vars = vars(PARAMCD)) %>%

Calculate ANRIND

derive_var_anrind()

Derive baseline flags

advs <- advs %>%

Calculate BASETYPE

derive_var_basetype( basetypes = rlang::exprs( "LAST Period 01" = !is.na(AVISIT), "NA" = is.na(AVISIT) ) ) %>%

Calculate ABLFL

restrict_derivation( derivation = derive_var_extreme_flag, args = params( by_vars = vars(STUDYID, USUBJID, PARAMCD), order = vars(ADTM, VSSPID, VSSEQ), new_var = ABLFL, mode = "last" ), filter = (!is.na(AVAL) & ADT < TRTSDT & !is.na(BASETYPE) & AVISIT == "BASELINE" ) )

Derive baseline information

advs <- advs %>%

Calculate BASE

derive_var_base( by_vars = vars(STUDYID, USUBJID, PARAMCD, BASETYPE), source_var = AVAL, new_var = BASE ) %>%

Calculate BASEC

derive_var_base( by_vars = vars(STUDYID, USUBJID, PARAMCD, BASETYPE), source_var = AVALC, new_var = BASEC ) %>%

Calculate BNRIND

derive_var_base( by_vars = vars(STUDYID, USUBJID, PARAMCD, BASETYPE), source_var = ANRIND, new_var = BNRIND ) %>%

Calculate CHG

derive_var_chg() %>%

Calculate PCHG

derive_var_pchg()

ANL01FL: Flag last result within an AVISIT and ATPT for post-baseline records

advs <- advs %>% restrict_derivation( derivation = derive_var_extreme_flag, args = params( new_var = ANL01FL, by_vars = vars(USUBJID, PARAMCD, AVISIT), order = vars(ADT, AVAL, VSSEQ), mode = "last" ), filter = !is.na(AVISITN) & ONTRTFL == "Y" )

Get treatment information

advs <- advs %>%

Assign TRTA, TRTP

mutate( TRTP = TRT01P, TRTA = TRT01A ) %>%

Create End of Treatment Record

restrict_derivation( derivation = derive_var_extreme_flag, args = params( by_vars = vars(STUDYID, USUBJID, PARAMCD), order = vars(ADT, VSSEQ), new_var = EOTFL, mode = "last" ), filter = (4 < VISITNUM & VISITNUM <= 13 & ANL01FL == "Y") ) %>% filter(EOTFL == "Y") %>% mutate( AVISIT = "End of Treatment", AVISITN = 99 ) %>% union_all(advs) %>% select(-EOTFL)

at this point TRTA and TRTP are still correct

Get ASEQ and AVALCATx and add PARAM/PARAMN

advs <- advs %>%

Calculate ASEQ

derive_var_obs_number( new_var = ASEQ, by_vars = vars(STUDYID, USUBJID), order = vars(PARAMCD, ADT, AVISITN, VISITNUM, VSSEQ), check_type = "error" )

after this section, the majority of TRTA and TRTP are NA

thomas-neitmann commented 2 years ago

@pharmaverse/admiral Can someone please take a closer look at this?

millerg23 commented 2 years ago

I think the issue is in the creation of "Create End of Treatment Record" when you do a union back with ADVS union_all(advs) %>%. The ADVS here, is from before TRTA and TRTP are created, so the only records with TRTA and TRTP populated, are for the records with AVISIT = "End of Treatment". If you move the creation of TRTA and TRTP to after the creation of "Create End of Treatment Record", it should be OK.

thomas-neitmann commented 2 years ago

Nice spot @millerg23! Indeed while in the mutate() both TRT variables are populated for all records, they are subsequently filtered and the unioned back with the ADVS from the chunk before wherethe TRT variables have not been created, yet. So @morrishj I don't think the issue is the derive_var_obs_number() step but rather what @millerg23 described. Anyhow, this is an error in our template which we have to fix.

bms63 commented 2 years ago

Shall we get this done for the release?

thomas-neitmann commented 2 years ago

Ideally, yes!

bms63 commented 2 years ago

@morrishj want to take this one on?

morrishj commented 2 years ago

Yep I can do this one!

On Thu, 11 Aug 2022 at 14:20, Ben Straub @.***> wrote:

@morrishj https://github.com/morrishj want to take this one on?

— Reply to this email directly, view it on GitHub https://github.com/pharmaverse/admiral/issues/1357#issuecomment-1211980921, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZUPJH62KXS5EGBTOGLEVFLVYT42ZANCNFSM56ADTIEA . You are receiving this because you were mentioned.Message ID: @.***>