[User feedback] Allow users to read data once if the data is used across multiple modules - reduce data loading time

[kumamiao]

From Chendi's reverse demo:

Research on how to only read a data once if a data (processed differently) is used across multiple modules. The use cases are quite frequent, for example, with current DDL data loading, if I were to use ADRS for swimlane plot and response analyses, ADRS data set needs to be read in twice, once as ADRSSWIM and once as ADRS because 2 data are pre-processed differently (example in Osprey sample app in the NEST docs, where ADRSSWIM uses individual parameters, and ADRS for response table uses overall parameters). Resolving this can help reduce data loading time.

[nikolas-burkoff]

This should be possible after https://github.com/insightsengineering/teal/issues/54 using a code_cdisc_dataset_connector

[gogonzo]

Edit: Issue can be closed - dependent vars can be used in code_dataset_connector and in the mutate_dataset.

# with mutate
adsl2 <- code_cdisc_dataset_connector(
  dataname = "ADSL2",
  "ADSL2 <- data.frame(STUDYID = 1, USIBJID = 1)", 
  keys = get_cdisc_keys("ADSL"),
  parent = character(0)
) %>%
  mutate_dataset(
    "ADSL2 <- ADSL",
    vars = list(ADSL = adsl)
  )

# with pull vars
adsl2 <- code_cdisc_dataset_connector(
  dataname = "ADSL2",
  "ADSL2 <- ADSL",
  keys = get_cdisc_keys("ADSL"),
  ADSL = adsl,
  parent = character(0)
)

Please check the example - ADSL2 is created based on the pulled ADSL. No need to pull twice

devtools::load_all()
adsl <- cdse_dataset_connector(dataname = "ADSL", cid = "cid6816691225664495616") %>%
  mutate_dataset("ADSL$USUBJID <- ADSL$UNI_TRNC")

adlb <- cdse_dataset_connector(
  dataname = "ADLB",
  cid = "cid6816690797501554688",
  keys = c("STUDYID", "USUBJID", "PARAMCD", "AVISIT", "LBDTC")
) %>%
  mutate_dataset(
    "ADLB$USUBJID <- ADLB$UNI_TRNC
    ADLB$dummy_sex <- ADSL$SEX[1]",
    vars = list(ADSL = adsl)
  )

adsl2 <- code_cdisc_dataset_connector(
  dataname = "ADSL2",
  "ADSL2 <- ADSL",
  keys = get_cdisc_keys("ADSL"),
  ADSL = adsl,
  parent = character(0)
)

data <- cdisc_data(
  cdse_data(
    connection = cdse_connection(env = "prod"),
    adsl,
    adlb
  ),
  adsl2
)

app <- init(
  data = data,
  modules = root_modules(
    teal.modules.general::tm_data_table()
  )
)

runApp(app)