Updates to upload functionality based on work in OhdsiSharing

[anthonysena]

Brings over the changes done in OHDSI/OhdsiSharing#8 for the upload_utility branch. Notes on updates:

• uploadResults now requires a folder instead of a zip file. A client calling this function should the function unzipResults which has been added to the package. Additionally, a new parameter called runCheckAndFixCommands has been added to uploadResults to steer people away from relying on these functions to fix data integrity issues.
• Renamed "fieldName" to "columnName" which has been used in nearly all of the Strategus modules. This will impact ComparatorSelectionExplorer so raising that for awareness.
• Settings files now use snake_case vs camelCase

Testing this with some recent Strategus results seemed to work. I'll also provide some annotations in this PR with some additional notes.

[anthonysena]

Also I wanted to confirm that I'm using RMM properly. Here is a snippet of code that I have:

# resultsFolder == a path to where Strategus results sit
# conn == connection to the results database
# resultsDatabseSchema == results database schema

# Create the results data model tables
sql <- ResultModelManager::generateSqlSchema(csvFilepath = file.path(resultsFolder, "resultsDataModelSpecification.csv"))
sql <- SqlRender::render(sql = sql, database_schema = resultsDatabaseSchema, warnOnMissingParameters = TRUE)
SqlRender::writeSql(sql, file.path(resultsFolder, "data_model.sql")) # Write the SQL for debugging
ResultModelManager::createResultsDataModel(connection = conn, schema = resultsDatabaseSchema, sql = sql)

# Upload the results
specification <- CohortGenerator::readCsv(file = file.path(resultsFolder, "resultsDataModelSpecification.csv"))
ResultModelManager::uploadResults(connection = conn, schema = resultsDatabaseSchema,  resultsFolder = resultsFolder, specifications = specification)

Just noting that in the SQL produced to create the tables, it is currently missing the primary keys which appears to be an issue with how the resultsDataModelSpecifications.csv (RDMS) files are encoded. Code like the following in SchemaGenerator.R will not work if the "primary_key" is not encoded exactly as 'yes' as shown here:

primaryKeyFields <- tableColumns[tableColumns$primaryKey == "yes", ]

[anthonysena]

Also noting that we should remove ParallelLogger for messages and instead consider rlang for messaging. ParallelLogger can pick up the messages when attached to a given run.

[codecov[bot]]

Codecov Report

:exclamation: No coverage uploaded for pull request base (upload_utility@5efa768). Click here to learn what that means. The diff coverage is n/a.

:exclamation: Current head 4a63f84 differs from pull request most recent head 28948bf. Consider uploading reports for the commit 28948bf to get more accurate results

@@                Coverage Diff                @@
##             upload_utility      #18   +/-   ##
=================================================
  Coverage                  ?   77.02%           
=================================================
  Files                     ?        5           
  Lines                     ?      631           
  Branches                  ?        0           
=================================================
  Hits                      ?      486           
  Misses                    ?      145           
  Partials                  ?        0           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.