Closed morrishj closed 2 years ago
@pharmaverse/admiral Can someone please take a closer look at this?
I think the issue is in the creation of "Create End of Treatment Record" when you do a union back with ADVS
union_all(advs) %>%
.
The ADVS here, is from before TRTA and TRTP are created, so the only records with TRTA and TRTP populated, are for the records with AVISIT = "End of Treatment".
If you move the creation of TRTA and TRTP to after the creation of "Create End of Treatment Record", it should be OK.
Nice spot @millerg23! Indeed while in the mutate()
both TRT
variables are populated for all records, they are subsequently filtered and the unioned back with the ADVS from the chunk before wherethe TRT
variables have not been created, yet. So @morrishj I don't think the issue is the derive_var_obs_number()
step but rather what @millerg23 described. Anyhow, this is an error in our template which we have to fix.
Shall we get this done for the release?
Ideally, yes!
@morrishj want to take this one on?
Yep I can do this one!
On Thu, 11 Aug 2022 at 14:20, Ben Straub @.***> wrote:
@morrishj https://github.com/morrishj want to take this one on?
— Reply to this email directly, view it on GitHub https://github.com/pharmaverse/admiral/issues/1357#issuecomment-1211980921, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZUPJH62KXS5EGBTOGLEVFLVYT42ZANCNFSM56ADTIEA . You are receiving this because you were mentioned.Message ID: @.***>
What happened?
TRTA and TRTP are derived correctly during the first half of the ADVS template, but when I get to the following section of code, both variables are changed and start showing large numbers of NA's.
advs <- advs %>% # Calculate ASEQ derive_var_obs_number( new_var = ASEQ, by_vars = vars(STUDYID, USUBJID), order = vars(PARAMCD, ADT, AVISITN, VISITNUM, VSSEQ), check_type = "error" )
Essentially, before I run that code, they are correct, and after I run that code, they are wrong.
As far as I can tell, that code shouldn't have any impact on TRTA and TRTP. TRT01A and TRT01P still stay the same.
I have cleared the workspace and restarted the session, and the same thing happens each time.
Session Information
Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0
locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8
[6] LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages: [1] stats graphics grDevices utils datasets methods base
other attached packages: [1] stringr_1.4.0 lubridate_1.8.0 admiral_0.7.0 dplyr_1.0.9
loaded via a namespace (and not attached): [1] rstudioapi_0.13 magrittr_2.0.3 hms_1.1.1 getPass_0.2-2 tidyselect_1.1.2 R6_2.5.1 rlang_1.0.3 fansi_1.0.3 httr_1.4.3
[10] tools_4.1.3 utf8_1.2.2 DBI_1.1.3 cli_3.3.0 ellipsis_0.3.2 assertthat_0.2.1 tibble_3.1.7 lifecycle_1.0.1 crayon_1.5.1
[19] tidyr_1.2.0 purrr_0.3.4 readr_2.1.2 tzdb_0.3.0 vctrs_0.4.1 curl_4.3.2 rice_3.0 glue_1.6.2 haven_2.5.0
[28] stringi_1.7.6 compiler_4.1.3 pillar_1.7.0 forcats_0.5.1 generics_0.1.2 jsonlite_1.8.0 pkgconfig_2.0.3
Reproducible Example
vs <- convert_blanks_to_na(vs)
---- Lookup tables ----
Assign PARAMCD, PARAM, and PARAMN
param_lookup <- tibble::tribble( ~VSTESTCD, ~PARAMCD, ~PARAM, ~PARAMN, "SYSBP", "SYSBP", "Systolic Blood Pressure (mmHg)", 1, "DIABP", "DIABP", "Diastolic Blood Pressure (mmHg)", 2, "PULSE", "PUL", "Pulse Rate (beats/min)", 3, "WEIGHT", "WGHT", "Weight (kg)", 4, "HEIGHT", "HGHT", "Height (cm)", 5, "TEMP", "TEMP", "Temperature (C)", 6, "RESP", "RESP", "Respiratory Rate (breaths/min)", 7, "OXYSAT", "OXYSAT", "Oxygen Saturation (%)", 8 ) attr(param_lookup$VSTESTCD, "label") <- "Vital Signs Test Short Name"
Assign ANRLO/HI, A1LO/HI
range_lookup <- tibble::tribble( ~PARAMCD, ~ANRLO, ~ANRHI, ~A1LO, ~A1HI, "SYSBP", 90, 130, 70, 140, "DIABP", 60, 80, 40, 90, "PUL", 60, 100, 40, 110, "TEMP", 36.5, 37.5, 35, 38, "RESP", 8, 20, NA, NA )
ASSIGN AVALCAT1
avalcat_lookup <- tibble::tribble( ~PARAMCD, ~AVALCA1N, ~AVALCAT1, "HGHT", 1, ">100 cm", "HGHT", 2, "<= 100 cm" )
---- User defined functions ----
Here are some examples of how you can create your own functions that
operates on vectors, which can be used in
mutate()
.format_avalcat1n <- function(param, aval) { case_when( param == "HGHT" & aval > 140 ~ 1, param == "HGHT" & aval <= 140 ~ 2 ) }
---- Derivations ----
Get list of ADSL vars required for derivations
adsl_vars <- vars(TRTSDT, TRTEDT, TRT01A, TRT01P, AP01SDTM)
advs <- vs %>%
Join ADSL with VS (need TRTSDT for ADY derivation)
derive_vars_merged( dataset_add = adsl, new_vars = adsl_vars, by_vars = vars(STUDYID, USUBJID) ) %>%
Calculate ADT, ADY
derive_vars_dt( new_vars_prefix = "A", dtc = VSDTC, flag_imputation = "none" ) %>%
calculate astdy and aendy
derive_vars_dy(reference_date = TRTSDT, source_vars = vars(ADT)) %>%
calculate adtm
derive_vars_dtm(new_vars_prefix = "A", dtc=VSDTC, flag_imputation="both")
advs <- advs %>%
Add PARAMCD only - add PARAM etc later
derive_vars_merged( dataset_add = param_lookup, new_vars = vars(PARAMCD), by_vars = vars(VSTESTCD) ) %>%
Calculate AVAL and AVALC
mutate( AVAL = VSSTRESN, AVALC = VSSTRESC, AVALU = VSSTRESU )
get visit info
advs <- advs %>%
Derive Timing
mutate( AVISIT = ifelse(!is.na(ADTM) & ADTM<= AP01SDTM, "BASELINE", VISIT) )
advs <- advs %>%
Derive Timing
mutate( AVISIT = ifelse(str_detect(VISIT,"(?i)Unscheduled"), "Unscheduled", AVISIT) )
advs <- advs %>%
Derive Timing
mutate( AVISITN = ifelse(AVISIT == "BASELINE", 0, VISITNUM) )
advs <- advs %>%
Calculate ONTRTFL
derive_var_ontrtfl( start_date = ADT, ref_start_date = TRTSDT, ref_end_date = TRTEDT, filter_pre_timepoint = AVISIT == "Baseline" )
Calculate ANRIND : requires the reference ranges ANRLO, ANRHI
Also accommodates the ranges A1LO, A1HI
advs <- advs %>% derive_vars_merged(dataset_add = range_lookup, by_vars = vars(PARAMCD)) %>%
Calculate ANRIND
derive_var_anrind()
Derive baseline flags
advs <- advs %>%
Calculate BASETYPE
derive_var_basetype( basetypes = rlang::exprs( "LAST Period 01" = !is.na(AVISIT), "NA" = is.na(AVISIT) ) ) %>%
Calculate ABLFL
restrict_derivation( derivation = derive_var_extreme_flag, args = params( by_vars = vars(STUDYID, USUBJID, PARAMCD), order = vars(ADTM, VSSPID, VSSEQ), new_var = ABLFL, mode = "last" ), filter = (!is.na(AVAL) & ADT < TRTSDT & !is.na(BASETYPE) & AVISIT == "BASELINE" ) )
Derive baseline information
advs <- advs %>%
Calculate BASE
derive_var_base( by_vars = vars(STUDYID, USUBJID, PARAMCD, BASETYPE), source_var = AVAL, new_var = BASE ) %>%
Calculate BASEC
derive_var_base( by_vars = vars(STUDYID, USUBJID, PARAMCD, BASETYPE), source_var = AVALC, new_var = BASEC ) %>%
Calculate BNRIND
derive_var_base( by_vars = vars(STUDYID, USUBJID, PARAMCD, BASETYPE), source_var = ANRIND, new_var = BNRIND ) %>%
Calculate CHG
derive_var_chg() %>%
Calculate PCHG
derive_var_pchg()
ANL01FL: Flag last result within an AVISIT and ATPT for post-baseline records
advs <- advs %>% restrict_derivation( derivation = derive_var_extreme_flag, args = params( new_var = ANL01FL, by_vars = vars(USUBJID, PARAMCD, AVISIT), order = vars(ADT, AVAL, VSSEQ), mode = "last" ), filter = !is.na(AVISITN) & ONTRTFL == "Y" )
Get treatment information
advs <- advs %>%
Assign TRTA, TRTP
mutate( TRTP = TRT01P, TRTA = TRT01A ) %>%
Create End of Treatment Record
restrict_derivation( derivation = derive_var_extreme_flag, args = params( by_vars = vars(STUDYID, USUBJID, PARAMCD), order = vars(ADT, VSSEQ), new_var = EOTFL, mode = "last" ), filter = (4 < VISITNUM & VISITNUM <= 13 & ANL01FL == "Y") ) %>% filter(EOTFL == "Y") %>% mutate( AVISIT = "End of Treatment", AVISITN = 99 ) %>% union_all(advs) %>% select(-EOTFL)
at this point TRTA and TRTP are still correct
Get ASEQ and AVALCATx and add PARAM/PARAMN
advs <- advs %>%
Calculate ASEQ
derive_var_obs_number( new_var = ASEQ, by_vars = vars(STUDYID, USUBJID), order = vars(PARAMCD, ADT, AVISITN, VISITNUM, VSSEQ), check_type = "error" )
after this section, the majority of TRTA and TRTP are NA