Error in prepareShinyData

sdebruyn commented 3 years ago

I tried to run the package as documented in the codeToRun and got the following error when trying to run prepareShinyData

> execute(
+     connectionDetails = connectionDetails,
+     cdmDatabaseSchema = cdmDatabaseSchema,
+     cohortDatabaseSchema = cohortDatabaseSchema,
+     cohortTable = cohortTable,
+     outputFolder = outputFolder,
+     databaseId = databaseId,
+     databaseName = databaseName,
+     databaseDescription = databaseDescription,
+     createCohorts = TRUE,
+     runExtraCharacterization = TRUE,
+     runAdHocChecks = TRUE,
+     runCohortDiagnostics = FALSE, # no need to run full cohort diagnostics for now
+     minCellCount = 5
+ )
Creating cohorts
Connecting using SQL Server driver
Creating cohort: tested_for_sarscov2
  |============================================================================================================| 100%
Executing SQL took 6.56 secs
Creating cohort: tested_positive_for_sarscov2
  |============================================================================================================| 100%
Executing SQL took 12 secs
Creating cohort: tested_with_a_covid_19_diagnosis_or_a_sarscov2_positive_test
  |============================================================================================================| 100%
Executing SQL took 14 secs
Creating cohort: with_a_covid_19_diagnosis_or_a_sarscov2_positive_test
  |============================================================================================================| 100%
Executing SQL took 3.8 secs
Creating cohort: with_a_covid_19_diagnosis_or_a_sarscov2_positive_test__deceased_
  |============================================================================================================| 100%
Executing SQL took 3.86 secs
Creating cohort: hospitalized_with_a_covid_19_diagnosis_or_a_sarscov2_positive_test
  |============================================================================================================| 100%
Executing SQL took 6.94 secs
Creating cohort: hospitalized_and_requiring_oxygen_with_a_covid_19_diagnosis_or_a_sarscov2_positive_test
  |============================================================================================================| 100%
Executing SQL took 15.3 secs
Creating cohort: hospitalized_and_requiring_intensive_services_with_a_covid_19_diagnosis_or_a_sarscov2_positive_test
  |============================================================================================================| 100%
Executing SQL took 10.5 secs
Creating cohort: hospitalized_and_requiring_ecmo_with_a_covid_19_diagnosis_or_a_sarscov2_positive_test
  |============================================================================================================| 100%
Executing SQL took 6.07 secs
Creating cohort: with_influenza_diagnosis_or_positive_test_2017_2018
  |============================================================================================================| 100%
Executing SQL took 4.72 secs
Creating cohort: with_influenza_diagnosis_or_positive_test_2017_2018__deaceased_
  |============================================================================================================| 100%
Executing SQL took 4.15 secs
Creating cohort: hospitalized_with_influenza_diagnosis_or_positive_test_2017_2018
  |============================================================================================================| 100%
Executing SQL took 3.97 secs
Creating cohort: hospitalized_with_influenza_diagnosis_or_positive_test_and_requiring_oxygen_2017_2018
  |============================================================================================================| 100%
Executing SQL took 6.4 secs
Creating cohort: hospitalized_with_influenza_diagnosis_or_positive_test_and_requiring_intensive_services_2017_2018
  |============================================================================================================| 100%
Executing SQL took 5.04 secs
Creating cohort: hospitalized_with_influenza_diagnosis_or_positive_test_and_ecmo_2017_2018
  |============================================================================================================| 100%
Executing SQL took 5.12 secs
  |============================================================================================================| 100%
Executing SQL took 0.113 secs
  |============================================================================================================| 100%
Executing SQL took 0.146 secs
  |============================================================================================================| 100%
Executing SQL took 0.174 secs
  |============================================================================================================| 100%
Executing SQL took 0.198 secs
Running characterization

── Column specification ──────────────────────────────────────────────────────────────────────────────────────────────
cols(
  cohortDefinitionId = col_double(),
  count = col_double(),
  cohortName = col_character()
)

Connecting using SQL Server driver
Sending temp tables to server
Constructing features on server
  |============================================================================================================| 100%
Executing SQL took 27 secs
Fetching data from server
Fetching data took 1.49 secs
Temporal Cohort characterization took 36.5 secs
Connecting using SQL Server driver
Sending temp tables to server
Constructing features on server
  |============================================================================================================| 100%
Executing SQL took 17 secs
Fetching data from server
Fetching data took 1.58 secs
Temporal Cohort characterization took 23.8 secs
Running database analysis
Connecting using SQL Server driver
Generating during life plausibility: CONDITION_OCCURRENCE
Generating during life plausibility: DRUG_EXPOSURE
Generating during life plausibility: MEASUREMENT
Generating table linkage counts: PERSON
Generating table linkage counts: CONDITION_OCCURRENCE
Generating table linkage counts: DRUG_EXPOSURE
Generating table linkage counts: MEASUREMENT
Generating table linkage counts: VISIT_OCCURRENCE
> EmaCovidFeasibility:::prepareShinyData(outputFolder, dsNames = NULL, shinyDataFolder = file.path(getwd(),outputFolder,"shinyData"))

── Column specification ──────────────────────────────────────────────────────────────────────────────────────────────
cols(
  cohortDefinitionId = col_double(),
  timeId = col_double(),
  covariateId = col_double(),
  mean = col_double(),
  averageValue = col_double(),
  sd = col_double(),
  startDayTemporalCharacterization = col_double(),
  endDayTemporalCharacterization = col_double()
)

── Column specification ──────────────────────────────────────────────────────────────────────────────────────────────
cols(
  covariateId = col_double(),
  covariateName = col_character(),
  analysisId = col_double(),
  conceptId = col_double()
)

Joining, by = "covariateId"

── Column specification ──────────────────────────────────────────────────────────────────────────────────────────────
cols(
  cohortDefinitionId = col_double(),
  count = col_double(),
  cohortName = col_character()
)

Joining, by = "cohortDefinitionId"
Joining, by = "cohortDefinitionId"

── Column specification ──────────────────────────────────────────────────────────────────────────────────────────────
cols(
  cohortDefinitionId = col_character(),
  timeId = col_character(),
  countValue = col_character(),
  covariateId = col_character(),
  averageValue = col_character(),
  sd = col_character(),
  minValue = col_character(),
  maxValue = col_character(),
  p10Value = col_character(),
  p25Value = col_character(),
  medianValue = col_character(),
  p75Value = col_character(),
  p90Value = col_character(),
  mean = col_character(),
  startDayTemporalCharacterization = col_character(),
  endDayTemporalCharacterization = col_character()
)

── Column specification ──────────────────────────────────────────────────────────────────────────────────────────────
cols(
  covariateId = col_double(),
  covariateName = col_character(),
  analysisId = col_double(),
  conceptId = col_double()
)

Joining, by = "covariateId"
Error: Can't join on `x$covariateId` x `y$covariateId` because of incompatible types.
ℹ `x$covariateId` is of type <character>>.
ℹ `y$covariateId` is of type <double>>.
Run `rlang::last_error()` to see where the error occurred.

ericaVoss commented 3 years ago

I think I ran into problems with this line too: EmaCovidFeasibility:::prepareShinyData(outputFolder, dsNames = NULL, shinyDataFolder = file.path(getwd(),outputFolder,"shinyData"))

I think this just does some work to make RShiny faster. Could you try skipping it and run the EmaCovidFeasibility:::launchEvidenceExplorer line instead.

sdebruyn commented 3 years ago

@ericaVoss Then you get an error about the missing data from the previous command

> EmaCovidFeasibility:::launchEvidenceExplorer(dataFolder = file.path(getwd(),outputFolder,"shinyData"))
Warning: cannot open compressed file '/home/rstudio/output/shinyData/prepared_cov_counts.rds', probable reason 'No such file or directory'
Error in gzfile(file, "rb") : cannot open the connection

ericaVoss commented 3 years ago

I think what is happening is when you pull the results some items are coming across as NA instead of counts or CONCEPT_IDs. I think this is causing the issue. @sdebruyn I'm happy to try to see how to fix the files with you.

ericaVoss commented 3 years ago

Okay, I can get passed the prepareShinyData() step, I added a 0 into continuousCovs.csv - covariateId column.

But now I'm getting an error on the launchEvidenceExplorer() - which happens at the shiny::runApp(appDir) step.

 Error in get_readable_names() :
  could not find function "get_readable_names"

ericaVoss commented 3 years ago

Never mind, if you run the library(EmaCovidFeasibility) statement it works.

emaprojects / EmaCovidFeasibility

Error in prepareShinyData #5