search

EHDEN / CatalogueExport

Exports the data from the OMOP-CDM that is necessary for the EHDEN Database Catalogue
Apache License 2.0
9 stars 6 forks source link

Error in scan (file = file, ... : line X did not have 6 elements #29

Closed scossin closed 3 years ago

scossin commented 3 years ago

Describe the bug https://github.com/EHDEN/CatalogueExport/blob/a7cdbd86e916e43ba1b67714236fc1eaf57b6320/R/CatalogueExport.R#L921 The error occurs if the log file doesn't contain 6 elements.

The log file doesn't contain 6 elements if only one of the queries doesn't work: an error is logged and the line added doesn't contain 6 elements: 

ParallelLogger::logError(sprintf("Analysis %d -- ERROR %s", mainSql$analysisId, e))

This error is very annoying because it occurs at the end of all the SQL queries before the CATALOGUE_RESULTS tables are created so it seems difficult to recover from this error.

To Reproduce Run an analysis with the CatalogueExport::catalogueExport() containing a SQL error in one of the queries, with default parameters and numThreads=1

PRijnbeek commented 3 years ago

Yes in principle no queries should fail and we may need to stop if they do to avoid these issues. Not sure about the 6 elements will try to reproduce (never happened earlier even when queries did fail for me).

PRijnbeek commented 3 years ago

I am not able to reproduce this when I simulate an error in the query on the current dev version

it then adds this row:

2021-03-07 17:07:20 [Main thread] ERROR CatalogueExport 3 Analysis 0 -- ERROR Error: Error executing SQL: com.microsoft.sqlserver.jdbc.SQLServerException: Invalid object name 'synpuf.personP'. An error report has been created at /Users/Rijnbeek/Documents/Github/EHDEN/CatalogueExport/errorReportSql.txt

Which is read successfully:

image
PRijnbeek commented 3 years ago

Will close it when pulling in dev. Re-open if this still is an issue.