In the following is the log-output after running the code:
(DKFZ_env) onko153-115:Roddy gerwin$ cat /Users/gerwin/.roddy/logs/default_240622_203128515.tsv
Roddy INFO Roddy version 3.3.1 NoExceptionThrown
Roddy INFO Timing initial checks: 89.995715 ms NoExceptionThrown
Roddy INFO Timing ftoggleini: 63.123059 ms NoExceptionThrown
Roddy INFO Timing initialize mode, print arguments: 63.941647 ms NoExceptionThrown
Roddy INFO [0] - testrun NoExceptionThrown
Roddy INFO [1] - WES_Vienna@WES NoExceptionThrown
Roddy INFO [2] - tumor NoExceptionThrown
Roddy INFO [3] - --useconfig=/Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/applicationProperties.ini NoExceptionThrown
Roddy INFO [4] - --useRoddyVersion=3.3.1 NoExceptionThrown
Roddy INFO [5] - --cvalues=fastq_list=tumor/tumor_R1.fastq.gz;tumor/tumor_R2.fastq.gz NoExceptionThrown
Roddy INFO [6] - --verbositylevel=5 NoExceptionThrown
Roddy INFO Timing initial: 12.575537 ms NoExceptionThrown
Roddy INFO Timing parseopt: 1.645531 ms NoExceptionThrown
Roddy INFO Loading properties file /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/applicationProperties.ini. NoExceptionThrown
Roddy INFO Timing loadprop: 11.39948 ms NoExceptionThrown
Roddy INFO Timing init proxy: 1.949407 ms NoExceptionThrown
de.dkfz.roddy.execution.io.fs.FileSystemAccessProvider INFO public static void initializeProvider(boolean fullSetup) NoExceptionThrown
Roddy INFO Timing init fsap: 52.11291 ms NoExceptionThrown
de.dkfz.roddy.execution.io.ExecutionService INFO public static void initializeService(boolean fullSetup) NoExceptionThrown
de.dkfz.roddy.execution.io.ExecutionService INFO Timing initialize exec service: 0.003398 ms NoExceptionThrown
Roddy INFO Timing init execserv: 22.0559 ms NoExceptionThrown
Roddy INFO public static void initializeFactory(boolean fullSetup) NoExceptionThrown
Roddy INFO Roddy will try to use the DirectSynchronousExecutionJobManager job manager class to manage jobs. NoExceptionThrown
Roddy INFO Timing init cmd fac: 81.779829 ms NoExceptionThrown
Roddy INFO Timing initserv: 26.759641 ms NoExceptionThrown
ConfigurationFactory CONFIG Searching for configuration files in: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/configurationfiles NoExceptionThrown
ConfigurationFactory CONFIG Searching for configuration files in: /Users/gerwin/.roddy/configurationFreeModeFiles NoExceptionThrown
ConfigurationFactory CONFIG Searching for configuration files in: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/configurationfiles NoExceptionThrown
ProjectLoader INFO Loading information configuration context for WES_Vienna from /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/configurationfiles/WES_Vienna.xml NoExceptionThrown
XSDValidator INFO Will validate configuration WES_Vienna. NoExceptionThrown
XSDValidator INFO Skipped configuration with id default, not available. NoExceptionThrown
LibrariesFactory INFO Parsing plugins folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins NoExceptionThrown
LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/.DS_Store NoExceptionThrown
LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows NoExceptionThrown
LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/COWorkflowsBasePlugin_1.2.0 NoExceptionThrown
LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/DefaultPlugin_1.2.2 NoExceptionThrown
LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/NativeWorkflowTestPlugin NoExceptionThrown
LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/PluginBase_1.2.0 NoExceptionThrown
LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/PluginBase_1.2.1 NoExceptionThrown
LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/Template NoExceptionThrown
LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/TestPluginWithJarFile NoExceptionThrown
LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/TestPluginWithoutJarFile NoExceptionThrown
LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows NoExceptionThrown
LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/COWorkflowsBasePlugin_1.2.0 NoExceptionThrown
LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/DefaultPlugin_1.2.2 NoExceptionThrown
LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/NativeWorkflowTestPlugin NoExceptionThrown
LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/PluginBase_1.2.0 NoExceptionThrown
LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/PluginBase_1.2.1 NoExceptionThrown
LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/Template NoExceptionThrown
LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/TestPluginWithJarFile NoExceptionThrown
LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/TestPluginWithoutJarFile NoExceptionThrown
LibrariesFactory INFO
Found tool folders:
bisulfiteWorkflow : /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/analysisTools/bisulfiteWorkflow
exomePipeline : /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/analysisTools/exomePipeline
qcPipeline : /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/analysisTools/qcPipeline
qcPipelineTools : /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/analysisTools/qcPipelineTools
roddyNativeTools : /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/DefaultPlugin_1.2.2/resources/analysisTools/roddyNativeTools
roddyTools : /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/DefaultPlugin_1.2.2/resources/analysisTools/roddyTools
tools : /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/COWorkflowsBasePlugin_1.2.0/resources/analysisTools/tools NoExceptionThrown
LibrariesFactory INFO
Loaded plugin AlignmentAndQCWorkflows:develop (/Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows)
Loaded plugin COWorkflowsBasePlugin:1.2.0-0 (/Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/COWorkflowsBasePlugin_1.2.0)
Loaded plugin PluginBase:1.2.1-0 (/Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/PluginBase_1.2.1)
Loaded plugin DefaultPlugin:1.2.2-0 (/Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/DefaultPlugin_1.2.2) NoExceptionThrown
ConfigurationFactory INFO Load configuration file /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/configurationfiles/WES_Vienna.xml NoExceptionThrown
ConfigurationFactory INFO Load configuration file /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/DefaultPlugin_1.2.2/resources/configurationFiles/default.xml NoExceptionThrown
ConfigurationFactory INFO Processing tool compressionDetection NoExceptionThrown
ConfigurationFactory INFO Processing tool defaultPluginLib NoExceptionThrown
ConfigurationFactory INFO Processing tool wrapinScript NoExceptionThrown
ConfigurationFactory INFO Load configuration file /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/configurationFiles/analysisExome.xml NoExceptionThrown
ConfigurationFactory INFO Processing tool onTargetCoverage NoExceptionThrown
ConfigurationFactory INFO Processing tool onTargetCoveragePlotter NoExceptionThrown
ConfigurationFactory INFO Processing tool onTargetCoveragePlotterBinary NoExceptionThrown
ConfigurationFactory INFO Processing tool compressionDetection NoExceptionThrown
ConfigurationFactory INFO Processing tool targetCoveragePerlScript NoExceptionThrown
ConfigurationFactory INFO Processing tool targetExtractCoverageSlim NoExceptionThrown
ConfigurationFactory INFO Processing tool annotateCovWindows NoExceptionThrown
ConfigurationFactory INFO Processing tool mergeAndFilterCovWindows NoExceptionThrown
ConfigurationFactory INFO Processing tool correctGc NoExceptionThrown
ConfigurationFactory INFO Load configuration file /Users/gerwin/Andreas/Pipeline3DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/configurationFiles/analysisQC.xml NoExceptionThrown
ConfigurationFactory SEVERE null: <filename pattern='${outputAnalysisBaseDirectory}/${coverageOutputDirectory}/${sourcefileAtomicPrefix,delimiter=""}.DepthOfCoverageGenome.txt' onMethod='BamFile.calcCoverage' class='CoverageTextFile'> NoExceptionThrown
ConfigurationFactory SEVERE null: <filename pattern='${outputAnalysisBaseDirectory}/${coverageOutputDirectory}/${sourcefileAtomicPrefix,delimiter=""}.DepthOfCoverage_RawBamFile.txt' onMethod='BamFile.rawBamCoverage' class='CoverageTextFile'> NoExceptionThrown
ConfigurationFactory SEVERE null: NoExceptionThrown
ConfigurationFactory INFO Processing tool workflowEnvironment_tbiLsf NoExceptionThrown
ConfigurationFactory INFO Processing tool workflowEnvironment_tbiPbs NoExceptionThrown
ConfigurationFactory INFO Processing tool workflowEnvironment_conda NoExceptionThrown
ConfigurationFactory INFO Processing tool bashLib NoExceptionThrown
ConfigurationFactory INFO Processing tool workflowLib NoExceptionThrown
ConfigurationFactory INFO Processing tool groupedGenomeCoverages NoExceptionThrown
ConfigurationFactory INFO Processing tool cleanupScript NoExceptionThrown
ConfigurationFactory INFO Processing tool fastqcClassify NoExceptionThrown
ConfigurationFactory INFO Processing tool fastqc NoExceptionThrown
ConfigurationFactory INFO Processing tool alignAndPairSlim NoExceptionThrown
ConfigurationFactory INFO Processing tool accelerated_alignAndPairSlim NoExceptionThrown
ConfigurationFactory INFO Processing tool commonAlignmentSettingsScript NoExceptionThrown
ConfigurationFactory INFO Processing tool coveragePlotScript NoExceptionThrown
ConfigurationFactory INFO Processing tool insertSizePlotScript NoExceptionThrown
ConfigurationFactory INFO Processing tool writeQcSummary NoExceptionThrown
ConfigurationFactory INFO Processing tool qcJson NoExceptionThrown
ConfigurationFactory INFO Processing tool sequencerDetection NoExceptionThrown
ConfigurationFactory INFO Processing tool convertIlluminaScores NoExceptionThrown
ConfigurationFactory INFO Processing tool genomeCoverage NoExceptionThrown
ConfigurationFactory INFO Processing tool plotDiffchrom NoExceptionThrown
ConfigurationFactory INFO Processing tool filterReadBins NoExceptionThrown
ConfigurationFactory INFO Processing tool coverageQcDImpl NoExceptionThrown
ConfigurationFactory INFO Processing tool genomeCoverageDImpl NoExceptionThrown
ConfigurationFactory INFO Processing tool collectBamMetrics NoExceptionThrown
ConfigurationFactory INFO Processing tool coveragePlot NoExceptionThrown
ConfigurationFactory INFO Processing tool coveragePlotSingle NoExceptionThrown
ConfigurationFactory INFO Processing tool fingerprint NoExceptionThrown
ConfigurationFactory INFO Processing tool mergeAndRemoveDuplicatesSlimPicard NoExceptionThrown
ConfigurationFactory INFO Processing tool mergeAndRemoveDuplicatesSlimBioBambam NoExceptionThrown
ConfigurationFactory INFO Processing tool mergeAndRemoveDuplicatesSlimSambamba NoExceptionThrown
ConfigurationFactory INFO Processing tool fakeDupmarkMetrics NoExceptionThrown
ConfigurationFactory INFO Processing tool printMissingReadGroups NoExceptionThrown
ConfigurationFactory INFO Processing tool combinedBamAnalysis NoExceptionThrown
ConfigurationFactory INFO Processing tool annotateCnvVcf NoExceptionThrown
ConfigurationFactory INFO Processing tool addMappability NoExceptionThrown
ConfigurationFactory INFO Processing tool estimateSex NoExceptionThrown
ConfigurationFactory INFO Processing tool mergeFilterCnv NoExceptionThrown
ConfigurationFactory INFO Processing tool correctGcBiasR NoExceptionThrown
ConfigurationFactory INFO Processing tool convertToJson NoExceptionThrown
ConfigurationFactory INFO Processing tool annotateCovWindows NoExceptionThrown
ConfigurationFactory INFO Processing tool mergeAndFilterCovWindows NoExceptionThrown
ConfigurationFactory INFO Processing tool correctGc NoExceptionThrown
ConfigurationFactory INFO Load configuration file /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/COWorkflowsBasePlugin_1.2.0/resources/configurationFiles/commonCOWorkflowsSettings.xml NoExceptionThrown
ConfigurationFactory INFO Processing tool analyzeBamHeader NoExceptionThrown
ConfigurationFactory INFO Processing tool checkBwaAndIndexVersions NoExceptionThrown
ConfigurationFactory INFO Processing tool annotateVcfFile NoExceptionThrown
ConfigurationFactory INFO Processing tool convertNewcolsToVcf NoExceptionThrown
ConfigurationFactory INFO Processing tool createpipes NoExceptionThrown
ConfigurationFactory INFO Processing tool headeredFileConcatenator NoExceptionThrown
ConfigurationFactory INFO Processing tool newColsToVcf NoExceptionThrown
ConfigurationFactory INFO Processing tool processAnnovar NoExceptionThrown
ConfigurationFactory INFO Processing tool seqContextAnnotator NoExceptionThrown
ConfigurationFactory INFO Processing tool vcfToAnnovar NoExceptionThrown
ConfigurationFactory INFO Processing tool bamIsComplete NoExceptionThrown
XSDValidator INFO Will validate configuration exomeAnalysis. NoExceptionThrown
XSDValidator INFO Will validate configuration qcAnalysis. NoExceptionThrown
ProjectLoader INFO Found project class: de.dkfz.roddy.core.Project NoExceptionThrown
RuntimeService WARNING Reading in jobs is not fully enabled! See RuntimeService readInExecutionContext(). The method does not reconstruct parent files and dependencies. NoExceptionThrown
ProjectLoader INFO Created an analysis object of class de.dkfz.roddy.core.Analysis. NoExceptionThrown
de.dkfz.roddy.execution.io.LocalExecutionService INFO de.dkfz.roddy.execution.io.MetadataTableFactory.getTable: Building metadata table from filesystem input is not implemented. NoExceptionThrown
RuntimeService INFO Looking for datasets in /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/Data_folder/data. NoExceptionThrown
RuntimeService INFO Found 2 datasets in /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/Data_folder/data. NoExceptionThrown
RuntimeService INFO Looking for datasets in /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/Data_folder/results. NoExceptionThrown
RuntimeService INFO Found 2 datasets in /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/Data_folder/results. NoExceptionThrown
RuntimeService INFO Found 2 datasets in the in- and output directories. NoExceptionThrown
RuntimeService INFO Selected dataset tumor for processing. NoExceptionThrown
Analysis INFO QUERY_STATUS NoExceptionThrown
Analysis INFO QUERY_STATUS NoExceptionThrown
de.dkfz.roddy.execution.io.fs.FileSystemAccessProvider INFO There was an error while trying to create file /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/Data_folder/results/tumor/roddyExecutionStore/exec_240622_203131504_gerwin_WES/jobStateLogfile.txt NoExceptionThrown
de.dkfz.roddy.execution.io.fs.FileSystemAccessProvider INFO There was an error while trying to load file /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/Data_folder/results/.roddyExecutionStore/zippedAnalysesMD5.txt NoExceptionThrown
Analysis INFO An unknown / unhandled exception occurred: 'Could not zip folder /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/analysisTools/exomePipeline to zip archive /Users/gerwin/.roddy/compressedAnalysisTools/cTools_AlignmentAndQCWorkflows:develop_exomePipeline.zip' NoExceptionThrown
Analysis INFO Could not zip folder /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/analysisTools/exomePipeline to zip archive /Users/gerwin/.roddy/compressedAnalysisTools/cTools_AlignmentAndQCWorkflows:develop_exomePipeline.zip
de.dkfz.roddy.tools.RoddyIOHelperMethods$NativeLinuxZipCompressor.compressDirectory(RoddyIOHelperMethods.groovy:86)
de.dkfz.roddy.tools.RoddyIOHelperMethods.compressDirectory(RoddyIOHelperMethods.groovy:222)
de.dkfz.roddy.tools.RoddyIOHelperMethods.compressDirectory(RoddyIOHelperMethods.groovy)
de.dkfz.roddy.execution.io.ExecutionService$_compressToolFolders_closure11.doCall(ExecutionService.groovy:710)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1027)
groovy.lang.Closure.call(Closure.java:414)
groovy.lang.Closure.call(Closure.java:430)
org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2030)
org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1890)
de.dkfz.roddy.execution.io.ExecutionService.compressToolFolders(ExecutionService.groovy:691)
de.dkfz.roddy.execution.io.ExecutionService.copyAnalysisToolsForContext(ExecutionService.groovy:629)
de.dkfz.roddy.execution.io.ExecutionService.writeFilesForExecution(ExecutionService.groovy:493)
de.dkfz.roddy.core.Analysis.prepareExecution(Analysis.java:364)
de.dkfz.roddy.core.Analysis.executeRun(Analysis.java:401)
de.dkfz.roddy.core.Analysis.run(Analysis.java:202)
de.dkfz.roddy.client.cliclient.RoddyCLIClient.testrun(RoddyCLIClient.groovy:524)
de.dkfz.roddy.client.cliclient.RoddyCLIClient.startMode(RoddyCLIClient.groovy:119)
de.dkfz.roddy.Roddy.parseRoddyStartupModeAndRun(Roddy.java:735)
de.dkfz.roddy.Roddy.startup(Roddy.java:287)
de.dkfz.roddy.Roddy.main(Roddy.java:214) NoExceptionThrown
Analysis INFO
There were errors for the execution context for dataset tumor
An uncaught error occurred during a run. SEVERE: Could not zip folder /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/analysisTools/exomePipeline to zip archive /Users/gerwin/.roddy/compressedAnalysisTools/cTools_AlignmentAndQCWorkflows:develop_exomePipeline.zip NoExceptionThrown
Analysis INFO
Please check extended logs in /Users/gerwin/.roddy/logs/default_240622_203128515.tsv for more details. NoExceptionThrown
Roddy INFO Timing parsemode: 2725.469005 ms NoExceptionThrown
Hello,
I want to run the AlignmentAndQCWorkflow for my tumor/normal FastQ-files (WES), but I get an error during the testrun. Here a part of my code:
FASTQ_LIST_TUMOR="tumor/tumor_R1.fastq.gz;tumor/tumor_R2.fastq.gz" FASTQ_LIST_NORMAL="normal/normal_R1.fastq.gz;normal/normal_R2.fastq.gz" echo "Running Roddy command for tumor dataset..." bash roddy.sh testrun WES_Vienna@WES tumor \ --useconfig=$main_directory/RoddyProject/Roddy/applicationProperties.ini \ --useRoddyVersion=3.3.1 \ --cvalues="fastq_list=$FASTQ_LIST_TUMOR" \ --verbositylevel=5
These are the XML and applicationProperties.ini files (for uploading converted into txt files): applicationProperties.txt WES_Vienna.txt
In the following is the log-output after running the code:
(DKFZ_env) onko153-115:Roddy gerwin$ cat /Users/gerwin/.roddy/logs/default_240622_203128515.tsv Roddy INFO Roddy version 3.3.1 NoExceptionThrown Roddy INFO Timing initial checks: 89.995715 ms NoExceptionThrown Roddy INFO Timing ftoggleini: 63.123059 ms NoExceptionThrown Roddy INFO Timing initialize mode, print arguments: 63.941647 ms NoExceptionThrown Roddy INFO [0] - testrun NoExceptionThrown Roddy INFO [1] - WES_Vienna@WES NoExceptionThrown Roddy INFO [2] - tumor NoExceptionThrown Roddy INFO [3] - --useconfig=/Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/applicationProperties.ini NoExceptionThrown Roddy INFO [4] - --useRoddyVersion=3.3.1 NoExceptionThrown Roddy INFO [5] - --cvalues=fastq_list=tumor/tumor_R1.fastq.gz;tumor/tumor_R2.fastq.gz NoExceptionThrown Roddy INFO [6] - --verbositylevel=5 NoExceptionThrown Roddy INFO Timing initial: 12.575537 ms NoExceptionThrown Roddy INFO Timing parseopt: 1.645531 ms NoExceptionThrown Roddy INFO Loading properties file /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/applicationProperties.ini. NoExceptionThrown Roddy INFO Timing loadprop: 11.39948 ms NoExceptionThrown Roddy INFO Timing init proxy: 1.949407 ms NoExceptionThrown de.dkfz.roddy.execution.io.fs.FileSystemAccessProvider INFO public static void initializeProvider(boolean fullSetup) NoExceptionThrown Roddy INFO Timing init fsap: 52.11291 ms NoExceptionThrown de.dkfz.roddy.execution.io.ExecutionService INFO public static void initializeService(boolean fullSetup) NoExceptionThrown de.dkfz.roddy.execution.io.ExecutionService INFO Timing initialize exec service: 0.003398 ms NoExceptionThrown Roddy INFO Timing init execserv: 22.0559 ms NoExceptionThrown Roddy INFO public static void initializeFactory(boolean fullSetup) NoExceptionThrown Roddy INFO Roddy will try to use the DirectSynchronousExecutionJobManager job manager class to manage jobs. NoExceptionThrown Roddy INFO Timing init cmd fac: 81.779829 ms NoExceptionThrown Roddy INFO Timing initserv: 26.759641 ms NoExceptionThrown ConfigurationFactory CONFIG Searching for configuration files in: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/configurationfiles NoExceptionThrown ConfigurationFactory CONFIG Searching for configuration files in: /Users/gerwin/.roddy/configurationFreeModeFiles NoExceptionThrown ConfigurationFactory CONFIG Searching for configuration files in: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/configurationfiles NoExceptionThrown ProjectLoader INFO Loading information configuration context for WES_Vienna from /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/configurationfiles/WES_Vienna.xml NoExceptionThrown XSDValidator INFO Will validate configuration WES_Vienna. NoExceptionThrown XSDValidator INFO Skipped configuration with id default, not available. NoExceptionThrown LibrariesFactory INFO Parsing plugins folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins NoExceptionThrown LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/.DS_Store NoExceptionThrown LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows NoExceptionThrown LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/COWorkflowsBasePlugin_1.2.0 NoExceptionThrown LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/DefaultPlugin_1.2.2 NoExceptionThrown LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/NativeWorkflowTestPlugin NoExceptionThrown LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/PluginBase_1.2.0 NoExceptionThrown LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/PluginBase_1.2.1 NoExceptionThrown LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/Template NoExceptionThrown LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/TestPluginWithJarFile NoExceptionThrown LibrariesFactory INFO Parsing plugin folder: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/TestPluginWithoutJarFile NoExceptionThrown LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows NoExceptionThrown LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/COWorkflowsBasePlugin_1.2.0 NoExceptionThrown LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/DefaultPlugin_1.2.2 NoExceptionThrown LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/NativeWorkflowTestPlugin NoExceptionThrown LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/PluginBase_1.2.0 NoExceptionThrown LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/PluginBase_1.2.1 NoExceptionThrown LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/Template NoExceptionThrown LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/TestPluginWithJarFile NoExceptionThrown LibrariesFactory INFO Processing plugin entry: /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/TestPluginWithoutJarFile NoExceptionThrown LibrariesFactory INFO Found tool folders: bisulfiteWorkflow : /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/analysisTools/bisulfiteWorkflow exomePipeline : /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/analysisTools/exomePipeline qcPipeline : /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/analysisTools/qcPipeline qcPipelineTools : /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/analysisTools/qcPipelineTools roddyNativeTools : /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/DefaultPlugin_1.2.2/resources/analysisTools/roddyNativeTools roddyTools : /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/DefaultPlugin_1.2.2/resources/analysisTools/roddyTools tools : /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/COWorkflowsBasePlugin_1.2.0/resources/analysisTools/tools NoExceptionThrown LibrariesFactory INFO Loaded plugin AlignmentAndQCWorkflows:develop (/Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows) Loaded plugin COWorkflowsBasePlugin:1.2.0-0 (/Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/COWorkflowsBasePlugin_1.2.0) Loaded plugin PluginBase:1.2.1-0 (/Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/PluginBase_1.2.1) Loaded plugin DefaultPlugin:1.2.2-0 (/Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/DefaultPlugin_1.2.2) NoExceptionThrown ConfigurationFactory INFO Load configuration file /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/configurationfiles/WES_Vienna.xml NoExceptionThrown ConfigurationFactory INFO Load configuration file /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/DefaultPlugin_1.2.2/resources/configurationFiles/default.xml NoExceptionThrown ConfigurationFactory INFO Processing tool compressionDetection NoExceptionThrown ConfigurationFactory INFO Processing tool defaultPluginLib NoExceptionThrown ConfigurationFactory INFO Processing tool wrapinScript NoExceptionThrown ConfigurationFactory INFO Load configuration file /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/configurationFiles/analysisExome.xml NoExceptionThrown ConfigurationFactory INFO Processing tool onTargetCoverage NoExceptionThrown ConfigurationFactory INFO Processing tool onTargetCoveragePlotter NoExceptionThrown ConfigurationFactory INFO Processing tool onTargetCoveragePlotterBinary NoExceptionThrown ConfigurationFactory INFO Processing tool compressionDetection NoExceptionThrown ConfigurationFactory INFO Processing tool targetCoveragePerlScript NoExceptionThrown ConfigurationFactory INFO Processing tool targetExtractCoverageSlim NoExceptionThrown ConfigurationFactory INFO Processing tool annotateCovWindows NoExceptionThrown ConfigurationFactory INFO Processing tool mergeAndFilterCovWindows NoExceptionThrown ConfigurationFactory INFO Processing tool correctGc NoExceptionThrown ConfigurationFactory INFO Load configuration file /Users/gerwin/Andreas/Pipeline3DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/configurationFiles/analysisQC.xml NoExceptionThrown ConfigurationFactory SEVERE null: <filename pattern='${outputAnalysisBaseDirectory}/${coverageOutputDirectory}/${sourcefileAtomicPrefix,delimiter=""}.DepthOfCoverageGenome.txt' onMethod='BamFile.calcCoverage' class='CoverageTextFile'> NoExceptionThrown ConfigurationFactory SEVERE null: <filename pattern='${outputAnalysisBaseDirectory}/${coverageOutputDirectory}/${sourcefileAtomicPrefix,delimiter=""}.DepthOfCoverage_RawBamFile.txt' onMethod='BamFile.rawBamCoverage' class='CoverageTextFile'> NoExceptionThrown ConfigurationFactory SEVERE null: NoExceptionThrown
ConfigurationFactory INFO Processing tool workflowEnvironment_tbiLsf NoExceptionThrown
ConfigurationFactory INFO Processing tool workflowEnvironment_tbiPbs NoExceptionThrown
ConfigurationFactory INFO Processing tool workflowEnvironment_conda NoExceptionThrown
ConfigurationFactory INFO Processing tool bashLib NoExceptionThrown
ConfigurationFactory INFO Processing tool workflowLib NoExceptionThrown
ConfigurationFactory INFO Processing tool groupedGenomeCoverages NoExceptionThrown
ConfigurationFactory INFO Processing tool cleanupScript NoExceptionThrown
ConfigurationFactory INFO Processing tool fastqcClassify NoExceptionThrown
ConfigurationFactory INFO Processing tool fastqc NoExceptionThrown
ConfigurationFactory INFO Processing tool alignAndPairSlim NoExceptionThrown
ConfigurationFactory INFO Processing tool accelerated_alignAndPairSlim NoExceptionThrown
ConfigurationFactory INFO Processing tool commonAlignmentSettingsScript NoExceptionThrown
ConfigurationFactory INFO Processing tool coveragePlotScript NoExceptionThrown
ConfigurationFactory INFO Processing tool insertSizePlotScript NoExceptionThrown
ConfigurationFactory INFO Processing tool writeQcSummary NoExceptionThrown
ConfigurationFactory INFO Processing tool qcJson NoExceptionThrown
ConfigurationFactory INFO Processing tool sequencerDetection NoExceptionThrown
ConfigurationFactory INFO Processing tool convertIlluminaScores NoExceptionThrown
ConfigurationFactory INFO Processing tool genomeCoverage NoExceptionThrown
ConfigurationFactory INFO Processing tool plotDiffchrom NoExceptionThrown
ConfigurationFactory INFO Processing tool filterReadBins NoExceptionThrown
ConfigurationFactory INFO Processing tool coverageQcDImpl NoExceptionThrown
ConfigurationFactory INFO Processing tool genomeCoverageDImpl NoExceptionThrown
ConfigurationFactory INFO Processing tool collectBamMetrics NoExceptionThrown
ConfigurationFactory INFO Processing tool coveragePlot NoExceptionThrown
ConfigurationFactory INFO Processing tool coveragePlotSingle NoExceptionThrown
ConfigurationFactory INFO Processing tool fingerprint NoExceptionThrown
ConfigurationFactory INFO Processing tool mergeAndRemoveDuplicatesSlimPicard NoExceptionThrown
ConfigurationFactory INFO Processing tool mergeAndRemoveDuplicatesSlimBioBambam NoExceptionThrown
ConfigurationFactory INFO Processing tool mergeAndRemoveDuplicatesSlimSambamba NoExceptionThrown
ConfigurationFactory INFO Processing tool fakeDupmarkMetrics NoExceptionThrown
ConfigurationFactory INFO Processing tool printMissingReadGroups NoExceptionThrown
ConfigurationFactory INFO Processing tool combinedBamAnalysis NoExceptionThrown
ConfigurationFactory INFO Processing tool annotateCnvVcf NoExceptionThrown
ConfigurationFactory INFO Processing tool addMappability NoExceptionThrown
ConfigurationFactory INFO Processing tool estimateSex NoExceptionThrown
ConfigurationFactory INFO Processing tool mergeFilterCnv NoExceptionThrown
ConfigurationFactory INFO Processing tool correctGcBiasR NoExceptionThrown
ConfigurationFactory INFO Processing tool convertToJson NoExceptionThrown
ConfigurationFactory INFO Processing tool annotateCovWindows NoExceptionThrown
ConfigurationFactory INFO Processing tool mergeAndFilterCovWindows NoExceptionThrown
ConfigurationFactory INFO Processing tool correctGc NoExceptionThrown
ConfigurationFactory INFO Load configuration file /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/COWorkflowsBasePlugin_1.2.0/resources/configurationFiles/commonCOWorkflowsSettings.xml NoExceptionThrown
ConfigurationFactory INFO Processing tool analyzeBamHeader NoExceptionThrown
ConfigurationFactory INFO Processing tool checkBwaAndIndexVersions NoExceptionThrown
ConfigurationFactory INFO Processing tool annotateVcfFile NoExceptionThrown
ConfigurationFactory INFO Processing tool convertNewcolsToVcf NoExceptionThrown
ConfigurationFactory INFO Processing tool createpipes NoExceptionThrown
ConfigurationFactory INFO Processing tool headeredFileConcatenator NoExceptionThrown
ConfigurationFactory INFO Processing tool newColsToVcf NoExceptionThrown
ConfigurationFactory INFO Processing tool processAnnovar NoExceptionThrown
ConfigurationFactory INFO Processing tool seqContextAnnotator NoExceptionThrown
ConfigurationFactory INFO Processing tool vcfToAnnovar NoExceptionThrown
ConfigurationFactory INFO Processing tool bamIsComplete NoExceptionThrown
XSDValidator INFO Will validate configuration exomeAnalysis. NoExceptionThrown
XSDValidator INFO Will validate configuration qcAnalysis. NoExceptionThrown
ProjectLoader INFO Found project class: de.dkfz.roddy.core.Project NoExceptionThrown
RuntimeService WARNING Reading in jobs is not fully enabled! See RuntimeService readInExecutionContext(). The method does not reconstruct parent files and dependencies. NoExceptionThrown
ProjectLoader INFO Created an analysis object of class de.dkfz.roddy.core.Analysis. NoExceptionThrown
de.dkfz.roddy.execution.io.LocalExecutionService INFO de.dkfz.roddy.execution.io.MetadataTableFactory.getTable: Building metadata table from filesystem input is not implemented. NoExceptionThrown
RuntimeService INFO Looking for datasets in /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/Data_folder/data. NoExceptionThrown
RuntimeService INFO Found 2 datasets in /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/Data_folder/data. NoExceptionThrown
RuntimeService INFO Looking for datasets in /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/Data_folder/results. NoExceptionThrown
RuntimeService INFO Found 2 datasets in /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/Data_folder/results. NoExceptionThrown
RuntimeService INFO Found 2 datasets in the in- and output directories. NoExceptionThrown
RuntimeService INFO Selected dataset tumor for processing. NoExceptionThrown
Analysis INFO QUERY_STATUS NoExceptionThrown
Analysis INFO QUERY_STATUS NoExceptionThrown
de.dkfz.roddy.execution.io.fs.FileSystemAccessProvider INFO There was an error while trying to create file /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/Data_folder/results/tumor/roddyExecutionStore/exec_240622_203131504_gerwin_WES/jobStateLogfile.txt NoExceptionThrown
de.dkfz.roddy.execution.io.fs.FileSystemAccessProvider INFO There was an error while trying to load file /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/Data_folder/results/.roddyExecutionStore/zippedAnalysesMD5.txt NoExceptionThrown
Analysis INFO An unknown / unhandled exception occurred: 'Could not zip folder /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/analysisTools/exomePipeline to zip archive /Users/gerwin/.roddy/compressedAnalysisTools/cTools_AlignmentAndQCWorkflows:develop_exomePipeline.zip' NoExceptionThrown
Analysis INFO Could not zip folder /Users/gerwin/Andreas/Pipeline3_DKFZ/RoddyProject/Roddy/dist/plugins/AlignmentAndQCWorkflows/resources/analysisTools/exomePipeline to zip archive /Users/gerwin/.roddy/compressedAnalysisTools/cTools_AlignmentAndQCWorkflows:develop_exomePipeline.zip
de.dkfz.roddy.tools.RoddyIOHelperMethods$NativeLinuxZipCompressor.compressDirectory(RoddyIOHelperMethods.groovy:86)
de.dkfz.roddy.tools.RoddyIOHelperMethods.compressDirectory(RoddyIOHelperMethods.groovy:222)
de.dkfz.roddy.tools.RoddyIOHelperMethods.compressDirectory(RoddyIOHelperMethods.groovy)
de.dkfz.roddy.execution.io.ExecutionService$_compressToolFolders_closure11.doCall(ExecutionService.groovy:710)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1027)
groovy.lang.Closure.call(Closure.java:414)
groovy.lang.Closure.call(Closure.java:430)
org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2030)
org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1890)
de.dkfz.roddy.execution.io.ExecutionService.compressToolFolders(ExecutionService.groovy:691)
de.dkfz.roddy.execution.io.ExecutionService.copyAnalysisToolsForContext(ExecutionService.groovy:629)
de.dkfz.roddy.execution.io.ExecutionService.writeFilesForExecution(ExecutionService.groovy:493)
de.dkfz.roddy.core.Analysis.prepareExecution(Analysis.java:364)
de.dkfz.roddy.core.Analysis.executeRun(Analysis.java:401)
de.dkfz.roddy.core.Analysis.run(Analysis.java:202)
de.dkfz.roddy.client.cliclient.RoddyCLIClient.testrun(RoddyCLIClient.groovy:524)
de.dkfz.roddy.client.cliclient.RoddyCLIClient.startMode(RoddyCLIClient.groovy:119)
de.dkfz.roddy.Roddy.parseRoddyStartupModeAndRun(Roddy.java:735)
de.dkfz.roddy.Roddy.startup(Roddy.java:287)
de.dkfz.roddy.Roddy.main(Roddy.java:214) NoExceptionThrown
Analysis INFO
There were errors for the execution context for dataset tumor