search

Illumina / Pisces

Somatic and germline variant caller for amplicon data. Recommended caller for tumor-only workflows.
GNU General Public License v3.0
94 stars 16 forks source link

error running GATK workflow on local machine #52

Open zahraj242 opened 4 years ago

zahraj242 commented 4 years ago

Hello friends

I would be extremely grateful if you could help me find the problem, I ran the following command inside docker GATK container in a local machine mac pro, and cannot find the " test.validation_.txt" file which is supposed to be the expected output in the execution directories. I have attached the commands

stderr1 Screen Shot 2020-07-16 at 12 00 30 AM stderr1 Screen Shot 2020-07-16 at 12 01 12 AM Screen Shot 2020-07-16 at 12 01 19 AM

and stderr+stdout+ validate-bam.wdl and validate-bam.inputs.json files

Thank you in advanced

Best

a) GATK version used : gatk-4.1.7.0 b) Exact GATK commands used c) The entire error log if applicable.

(gatk) root@bc3c6aca6231:/gatk/my_data/tools# java -jar cromwell-51.jar run /gatk/my_data/seq-format-validation/validate-bam.wdl --inputs /gatk/my_data/seq-format-validation/validate-bam.inputs.json

[2020-07-14 05:09:22,78] [info] Running with database db.url = jdbc:hsqldb:mem:f10b64bd-d8ca-4428-917b-311fca24c372;shutdown=false;hsqldb.tx=mvcc

[2020-07-14 05:09:29,36] [info] Running migration RenameWorkflowOptionsInMetadata with a read batch size of 100000 and a write batch size of 100000

[2020-07-14 05:09:29,37] [info] [RenameWorkflowOptionsInMetadata] 100%

[2020-07-14 05:09:29,47] [info] Running with database db.url = jdbc:hsqldb:mem:e337a356-2f0c-4389-92c5-255465180f24;shutdown=false;hsqldb.tx=mvcc

[2020-07-14 05:09:29,89] [info] Slf4jLogger started

[2020-07-14 05:09:30,10] [info] Workflow heartbeat configuration:

{

"cromwellId" : "cromid-ca5c695",

"heartbeatInterval" : "2 minutes",

"ttl" : "10 minutes",

"failureShutdownDuration" : "5 minutes",

"writeBatchSize" : 10000,

"writeThreshold" : 10000

}

[2020-07-14 05:09:30,23] [info] Metadata summary refreshing every 1 second.

[2020-07-14 05:09:30,23] [warn] 'docker.hash-lookup.gcr-api-queries-per-100-seconds' is being deprecated, use 'docker.hash-lookup.gcr.throttle' instead (see reference.conf)

[2020-07-14 05:09:30,25] [info] CallCacheWriteActor configured to flush with batch size 100 and process rate 3 seconds.

[2020-07-14 05:09:30,26] [info] KvWriteActor configured to flush with batch size 200 and process rate 5 seconds.

[2020-07-14 05:09:30,26] [info] WriteMetadataActor configured to flush with batch size 200 and process rate 5 seconds.

[2020-07-14 05:09:30,36] [info] JobExecutionTokenDispenser - Distribution rate: 50 per 1 seconds.

[2020-07-14 05:09:30,46] [info] SingleWorkflowRunnerActor: Version 51

[2020-07-14 05:09:30,48] [info] SingleWorkflowRunnerActor: Submitting workflow

[2020-07-14 05:09:30,55] [info] Unspecified type (Unspecified version) workflow 968be82c-eef3-4bdb-a1ab-3d4e2ca70674 submitted

[2020-07-14 05:09:30,66] [info] SingleWorkflowRunnerActor: Workflow submitted 968be82c-eef3-4bdb-a1ab-3d4e2ca70674

[2020-07-14 05:09:30,67] [info] 1 new workflows fetched by cromid-ca5c695: 968be82c-eef3-4bdb-a1ab-3d4e2ca70674

[2020-07-14 05:09:30,68] [info] WorkflowManagerActor Starting workflow 968be82c-eef3-4bdb-a1ab-3d4e2ca70674

[2020-07-14 05:09:30,69] [info] WorkflowManagerActor Successfully started WorkflowActor-968be82c-eef3-4bdb-a1ab-3d4e2ca70674

[2020-07-14 05:09:30,69] [info] Retrieved 1 workflows from the WorkflowStoreActor

[2020-07-14 05:09:30,72] [info] WorkflowStoreHeartbeatWriteActor configured to flush with batch size 10000 and process rate 2 minutes.

[2020-07-14 05:09:30,83] [info] MaterializeWorkflowDescriptorActor [968be82c]: Parsing workflow as WDL 1.0

[2020-07-14 05:09:31,60] [info] MaterializeWorkflowDescriptorActor [968be82c]: Call-to-Backend assignments: ValidateBamsWf.ValidateBAM -> Local

[2020-07-14 05:09:31,82] [warn] Local [968be82c]: Key/s [memory, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.

[2020-07-14 05:09:35,38] [info] Not triggering log of token queue status. Effective log interval = None

[2020-07-14 05:09:37,15] [info] WorkflowExecutionActor-968be82c-eef3-4bdb-a1ab-3d4e2ca70674 [968be82c]: Starting ValidateBamsWf.ValidateBAM

[2020-07-14 05:09:37,39] [info] Assigned new job execution tokens to the following groups: 968be82c: 1

[2020-07-14 05:09:41,61] [warn] BackgroundConfigAsyncJobExecutionActor [968be82cValidateBamsWf.ValidateBAM:0:1]: Unrecognized runtime attribute keys: disks, memory

[2020-07-14 05:09:41,71] [info] BackgroundConfigAsyncJobExecutionActor [968be82cValidateBamsWf.ValidateBAM:0:1]: /gatk/gatk \

ValidateSamFile \

--INPUT /cromwell-executions/ValidateBamsWf/968be82c-eef3-4bdb-a1ab-3d4e2ca70674/call-ValidateBAM/shard-0/inputs/-1942028726/test.bam \

--OUTPUT test.validation_.txt \

--MODE SUMMARY

[2020-07-14 05:09:41,76] [info] BackgroundConfigAsyncJobExecutionActor [968be82cValidateBamsWf.ValidateBAM:0:1]: executing: # make sure there is no preexisting Docker CID file

rm -f /gatk/my_data/tools/cromwell-executions/ValidateBamsWf/968be82c-eef3-4bdb-a1ab-3d4e2ca70674/call-ValidateBAM/shard-0/execution/docker_cid

run as in the original configuration without --rm flag (will remove later)

docker run \

--cidfile /gatk/my_data/tools/cromwell-executions/ValidateBamsWf/968be82c-eef3-4bdb-a1ab-3d4e2ca70674/call-ValidateBAM/shard-0/execution/docker_cid \

-i \

\

--entrypoint /bin/bash \

-v /gatk/my_data/tools/cromwell-executions/ValidateBamsWf/968be82c-eef3-4bdb-a1ab-3d4e2ca70674/call-ValidateBAM/shard-0:/cromwell-executions/ValidateBamsWf/968be82c-eef3-4bdb-a1ab-3d4e2ca70674/call-ValidateBAM/shard-0:delegated \

broadinstitute/gatk@sha256:18146e79d06787483310e5de666502090a480e10ac0fad06a36a5e7a5c9bb1dc /cromwell-executions/ValidateBamsWf/968be82c-eef3-4bdb-a1ab-3d4e2ca70674/call-ValidateBAM/shard-0/execution/script

get the return code (working even if the container was detached)

rc=$(docker wait cat /gatk/my_data/tools/cromwell-executions/ValidateBamsWf/968be82c-eef3-4bdb-a1ab-3d4e2ca70674/call-ValidateBAM/shard-0/execution/docker_cid)

remove the container after waiting

docker rm cat /gatk/my_data/tools/cromwell-executions/ValidateBamsWf/968be82c-eef3-4bdb-a1ab-3d4e2ca70674/call-ValidateBAM/shard-0/execution/docker_cid

return exit code

exit $rc

[2020-07-14 05:09:45,29] [info] BackgroundConfigAsyncJobExecutionActor [968be82cValidateBamsWf.ValidateBAM:0:1]: job id: 243

[2020-07-14 05:09:45,29] [info] BackgroundConfigAsyncJobExecutionActor [968be82cValidateBamsWf.ValidateBAM:0:1]: Status change from - to Done

[2020-07-14 05:09:46,38] [info] WorkflowManagerActor Workflow 968be82c-eef3-4bdb-a1ab-3d4e2ca70674 failed (during ExecutingWorkflowState): Job ValidateBamsWf.ValidateBAM:0:1 exited with return code -1 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details.

Check the content of stderr for potential additional information: /gatk/my_data/tools/cromwell-executions/ValidateBamsWf/968be82c-eef3-4bdb-a1ab-3d4e2ca70674/call-ValidateBAM/shard-0/execution/stderr.

Could not retrieve content: Could not read from /gatk/my_data/tools/cromwell-executions/ValidateBamsWf/968be82c-eef3-4bdb-a1ab-3d4e2ca70674/call-ValidateBAM/shard-0/execution/stderr: /gatk/my_data/tools/cromwell-executions/ValidateBamsWf/968be82c-eef3-4bdb-a1ab-3d4e2ca70674/call-ValidateBAM/shard-0/execution/stderr

[2020-07-14 05:09:46,38] [info] WorkflowManagerActor WorkflowActor-968be82c-eef3-4bdb-a1ab-3d4e2ca70674 is in a terminal state: WorkflowFailedState

[2020-07-14 05:09:51,97] [info] SingleWorkflowRunnerActor workflow finished with status 'Failed'.

[2020-07-14 05:09:55,28] [info] Workflow polling stopped

[2020-07-14 05:09:55,30] [info] 0 workflows released by cromid-ca5c695

[2020-07-14 05:09:55,30] [info] Shutting down WorkflowStoreActor - Timeout = 5 seconds

[2020-07-14 05:09:55,30] [info] Shutting down WorkflowLogCopyRouter - Timeout = 5 seconds

[2020-07-14 05:09:55,30] [info] Shutting down JobExecutionTokenDispenser - Timeout = 5 seconds

[2020-07-14 05:09:55,31] [info] JobExecutionTokenDispenser stopped

[2020-07-14 05:09:55,31] [info] Aborting all running workflows.

[2020-07-14 05:09:55,31] [info] WorkflowStoreActor stopped

[2020-07-14 05:09:55,31] [info] WorkflowLogCopyRouter stopped

[2020-07-14 05:09:55,31] [info] Shutting down WorkflowManagerActor - Timeout = 3600 seconds

[2020-07-14 05:09:55,32] [info] WorkflowManagerActor All workflows finished

[2020-07-14 05:09:55,32] [info] WorkflowManagerActor stopped

[2020-07-14 05:09:55,53] [info] Connection pools shut down

[2020-07-14 05:09:55,53] [info] Shutting down SubWorkflowStoreActor - Timeout = 1800 seconds

[2020-07-14 05:09:55,53] [info] Shutting down JobStoreActor - Timeout = 1800 seconds

[2020-07-14 05:09:55,53] [info] SubWorkflowStoreActor stopped

[2020-07-14 05:09:55,54] [info] JobStoreActor stopped

[2020-07-14 05:09:55,53] [info] Shutting down CallCacheWriteActor - Timeout = 1800 seconds

[2020-07-14 05:09:55,54] [info] Shutting down ServiceRegistryActor - Timeout = 1800 seconds

[2020-07-14 05:09:55,54] [info] CallCacheWriteActor Shutting down: 0 queued messages to process

[2020-07-14 05:09:55,54] [info] CallCacheWriteActor stopped

[2020-07-14 05:09:55,54] [info] Shutting down DockerHashActor - Timeout = 1800 seconds

[2020-07-14 05:09:55,54] [info] Shutting down IoProxy - Timeout = 1800 seconds

[2020-07-14 05:09:55,54] [info] IoProxy stopped

[2020-07-14 05:09:55,54] [info] DockerHashActor stopped

[2020-07-14 05:09:55,55] [info] Shutting down connection pool: curAllocated=1 idleQueues.size=1 waitQueue.size=0 maxWaitQueueLimit=256 closed=false

[2020-07-14 05:09:55,55] [info] Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false

[2020-07-14 05:09:55,55] [info] WriteMetadataActor Shutting down: 0 queued messages to process

[2020-07-14 05:09:55,55] [info] Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false

[2020-07-14 05:09:55,55] [info] KvWriteActor Shutting down: 0 queued messages to process

[2020-07-14 05:09:55,55] [info] Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false

[2020-07-14 05:09:55,55] [info] ServiceRegistryActor stopped

[2020-07-14 05:09:55,58] [info] Database closed

[2020-07-14 05:09:55,58] [info] Stream materializer shut down

[2020-07-14 05:09:55,58] [info] WDL HTTP import resolver closed

Workflow 968be82c-eef3-4bdb-a1ab-3d4e2ca70674 transitioned to state Failed