OHDSI / CohortDiagnostics

An R package for performing various cohort diagnostics.
https://ohdsi.github.io/CohortDiagnostics
40 stars 45 forks source link

Error in formatChunk(pkValuesInDb = env$primaryKeyValuesInDb, chunk = chunk) : concept_set_id is of type numeric which cannot be converted between data frames pkValuesInDb and chunk #1121

Closed ablack3 closed 1 week ago

ablack3 commented 3 weeks ago

I'm getting the following error when I run CohortDiagnostics::createMergedResultsFile(dataFolder = exportFolder, overwrite = TRUE)

Error in formatChunk(pkValuesInDb = env$primaryKeyValuesInDb, chunk = chunk) : 
  concept_set_id is of type numeric which cannot be converted between data frames pkValuesInDb and chunk

The issue seems to be with the concept_sets.csv table and coming from the fomatChunk in ResultsModelManger function.

I'm guessing that when the file is read in the column type for concept_set_id is numeric instead of integer but I don't really know yet where the issue is.

Here is the traceback

> traceback()
13: stop(errorMsg) at DataModel.R#824
12: formatChunk(pkValuesInDb = env$primaryKeyValuesInDb, chunk = chunk) at DataModel.R#350
11: uploadChunk(chunk, pos, env, specifications, resultsFolder, connection, 
        runCheckAndFixCommands, forceOverWriteOfSpecifications) at DataModel.R#481
10: private$callback(data, index) at callback.R#82
9: (function (data, index) 
   {
       result <- private$callback(data, index)
       private$cancel <- identical(result, FALSE)
   })(structure(list(cohort_id = 568, concept_set_id = 1, concept_set_sql = "SELECT 1 as codeset_id, c.concept_id FROM (select distinct I.concept_id ...... (deleted this for brevity)
8: read_tokens_chunked_(data, callback, chunk_size, tokenizer, col_specs, 
       col_names, locale_, spec, progress) at read_delim_chunked.R#41
7: read_tokens_chunked(ds, callback = callback, chunk_size = chunk_size, 
       spec = spec, tokenizer, spec$cols, names(spec$cols), locale_ = locale, 
       progress = progress)
6: read_delimited_chunked(file, callback = callback, chunk_size = chunk_size, 
       tokenizer = tokenizer, col_names = col_names, col_types = col_types, 
       locale = locale, skip = skip, skip_empty_rows = skip_empty_rows, 
       comment = comment, guess_max = guess_max, progress = progress, 
       show_col_types = show_col_types) at read_delim_chunked.R#101
5: readr::read_csv_chunked(file = file.path(resultsFolder, csvFileName), 
       callback = function(chunk, pos) uploadChunk(chunk, pos, env, 
           specifications, resultsFolder, connection, runCheckAndFixCommands, 
           forceOverWriteOfSpecifications), chunk_size = 1e+07, 
       col_types = readr::cols(), guess_max = 1e+06, progress = FALSE) at DataModel.R#481
4: uploadTable(tableName, connection, schema, tablePrefix, databaseId, 
       resultsFolder, specifications, runCheckAndFixCommands, forceOverWriteOfSpecifications, 
       purgeSiteDataBeforeUploading, warnOnMissingTable) at DataModel.R#609
3: ResultModelManager::uploadResults(connectionDetails = connectionDetails, 
       schema = schema, resultsFolder = unzipFolder, tablePrefix = tablePrefix, 
       forceOverWriteOfSpecifications = forceOverWriteOfSpecifications, 
       purgeSiteDataBeforeUploading = purgeSiteDataBeforeUploading, 
       runCheckAndFixCommands = TRUE, databaseIdentifierFile = "database.csv", 
       specifications = getResultsDataModelSpecifications(), warnOnMissingTable = FALSE, 
       ...) at ResultsDataModel.R#136
2: uploadResults(connectionDetails = connectionDetails, schema = "main", 
       zipFileName = zipFileName, tablePrefix = tablePrefix) at Shiny.R#202
1: CohortDiagnostics::createMergedResultsFile(dataFolder = exportFolder, 
       overwrite = TRUE)
MaximMoinat commented 1 week ago

Encountered the same issue with ResultsModelManager v0.5.7, but was solved after updating to the latest version. Seems to be fixed now in v0.5.8 through this commit.

The issue can be closed.