brianjohnhaas
commented
2 years ago Hi,
Sorry about this. Can you try updating the import statement at the top of:
WDL/ctat_mutations.wdl
from:
import " https://raw.githubusercontent.com/NCIP/ctat-mutations/Terra-3.2.0/WDL/subworkflows/annotate_variants.wdl" as VariantAnnotation
to more simply:
import "subworkflows/annotate_variants.wdl" as VariantAnnotation
Let's see if that works for you.
best,
~brian
On Mon, Nov 29, 2021 at 3:20 AM Jianbiaoli @.***> wrote:
Hi,
I tried to run CTAT-mutations using. My command is:
/path1/singularity exec --bind /path2/ctat_genome_lib_build_dir,/path4/ /path3/ctat_mutations.v3.2.0.simg /usr/local/src/ctat-mutations/ctat_mutations --genome_lib_dir /path2/ctat_genome_lib_build_dir --cpu 4 --boosting_method none --sample_id RHN22 --bam /path4/RHN22.star.Aligned.sortedByCoord.out.bam -O /path4/mutations
Then I got the error in the log file:
[2021-11-29 07:02:02,91] [info] Metadata summary refreshing every 1 second. [2021-11-29 07:02:02,97] [info] KvWriteActor configured to flush with batch size 200 and process rate 5 seconds. [2021-11-29 07:02:02,97] [info] WriteMetadataActor configured to flush with batch size 200 and process rate 5 seconds. [2021-11-29 07:02:03,00] [warn] 'docker.hash-lookup.gcr-api-queries-per-100-seconds' is being deprecated, use 'docker.hash-lookup.gcr.throttle' instead (see reference.conf) [2021-11-29 07:02:03,00] [info] CallCacheWriteActor configured to flush with batch size 100 and process rate 3 seconds. [2021-11-29 07:02:03,14] [info] JobExecutionTokenDispenser - Distribution rate: 50 per 1 seconds. [2021-11-29 07:02:03,24] [info] SingleWorkflowRunnerActor: Version 58 [2021-11-29 07:02:03,26] [info] SingleWorkflowRunnerActor: Submitting workflow [2021-11-29 07:02:03,38] [info] Unspecified type (Unspecified version) workflow 3547ffb0-3397-4423-b6b9-18edb10fe366 submitted [2021-11-29 07:02:03,42] [info] SingleWorkflowRunnerActor: Workflow submitted 3547ffb0-3397-4423-b6b9-18edb10fe366 [2021-11-29 07:02:03,43] [info] 1 new workflows fetched by cromid-e9c15e1: 3547ffb0-3397-4423-b6b9-18edb10fe366 [2021-11-29 07:02:03,45] [info] WorkflowManagerActor Starting workflow 3547ffb0-3397-4423-b6b9-18edb10fe366 [2021-11-29 07:02:03,46] [info] WorkflowManagerActor Successfully started WorkflowActor-3547ffb0-3397-4423-b6b9-18edb10fe366 [2021-11-29 07:02:03,46] [info] Retrieved 1 workflows from the WorkflowStoreActor [2021-11-29 07:02:03,49] [info] WorkflowStoreHeartbeatWriteActor configured to flush with batch size 10000 and process rate 2 minutes. [2021-11-29 07:02:03,65] [info] MaterializeWorkflowDescriptorActor [3547ffb0]: Parsing workflow as WDL 1.0 [2021-11-29 07:02:08,15] [info] Not triggering log of token queue status. Effective log interval = None [2021-11-29 07:02:20,42] [info] WorkflowManagerActor Workflow 3547ffb0-3397-4423-b6b9-18edb10fe366 failed (during MaterializingWorkflowDescriptorState): cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor$$anon$1: Workflow input processing failed: Failed to import 'https://raw.githubusercontent.com/NCIP/ctat-mutations/Terra-3.2.0/WDL/subworkflows/annotate_variants.wdl' (reason 1 of 3): Failed to resolve 'https://raw.githubusercontent.com/NCIP/ctat-mutations/Terra-3.2.0/WDL/subworkflows/annotate_variants.wdl' using resolver: 'relative to directory [...]/WDL (escaping allowed)' (reason 1 of 1): File not found: https://raw.githubusercontent.com/NCIP/ctat-mutations/Terra-3.2.0/WDL/subworkflows/annotate_variants.wdl Failed to import 'https://raw.githubusercontent.com/NCIP/ctat-mutations/Terra-3.2.0/WDL/subworkflows/annotate_variants.wdl' (reason 2 of 3): Failed to resolve 'https://raw.githubusercontent.com/NCIP/ctat-mutations/Terra-3.2.0/WDL/subworkflows/annotate_variants.wdl' using resolver: 'entire local filesystem (relative to '/')' (reason 1 of 1): File not found: https://raw.githubusercontent.com/NCIP/ctat-mutations/Terra-3.2.0/WDL/subworkflows/annotate_variants.wdl Failed to import 'https://raw.githubusercontent.com/NCIP/ctat-mutations/Terra-3.2.0/WDL/subworkflows/annotate_variants.wdl' (reason 3 of 3): Failed to resolve 'https://raw.githubusercontent.com/NCIP/ctat-mutations/Terra-3.2.0/WDL/subworkflows/annotate_variants.wdl' using resolver: 'http importer (no 'relative-to' origin)' (reason 1 of 1): Failed to download https://raw.githubusercontent.com/NCIP/ctat-mutations/Terra-3.2.0/WDL/subworkflows/annotate_variants.wdl (reason 1 of 1): HTTP resolver with headers had an unexpected error (Futures timed out after [15 seconds]) at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor.cromwell$engine$workflow$lifecycle$materialization$MaterializeWorkflowDescriptorActor$$workflowInitializationFailed(MaterializeWorkflowDescriptorActor.scala:257) at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor$$anonfun$2.applyOrElse(MaterializeWorkflowDescriptorActor.scala:227) at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor$$anonfun$2.applyOrElse(MaterializeWorkflowDescriptorActor.scala:222) at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:38) at akka.actor.FSM.processEvent(FSM.scala:707) at akka.actor.FSM.processEvent$(FSM.scala:704) at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor.akka$actor$LoggingFSM$$super$processEvent(MaterializeWorkflowDescriptorActor.scala:169) at akka.actor.LoggingFSM.processEvent(FSM.scala:847) at akka.actor.LoggingFSM.processEvent$(FSM.scala:829) at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor.processEvent(MaterializeWorkflowDescriptorActor.scala:169) at akka.actor.FSM.akka$actor$FSM$$processMsg(FSM.scala:701) at akka.actor.FSM$$anonfun$receive$1.applyOrElse(FSM.scala:695) at akka.actor.Actor.aroundReceive(Actor.scala:539) at akka.actor.Actor.aroundReceive$(Actor.scala:537) at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor.aroundReceive(MaterializeWorkflowDescriptorActor.scala:169) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:614) at akka.actor.ActorCell.invoke(ActorCell.scala:583) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:268) at akka.dispatch.Mailbox.run(Mailbox.scala:229) at akka.dispatch.Mailbox.exec(Mailbox.scala:241) at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
[2021-11-29 07:02:20,43] [info] WorkflowManagerActor WorkflowActor-3547ffb0-3397-4423-b6b9-18edb10fe366 is in a terminal state: WorkflowFailedState [2021-11-29 07:02:24,27] [info] SingleWorkflowRunnerActor workflow finished with status 'Failed'. [2021-11-29 07:02:28,08] [info] SingleWorkflowRunnerActor writing metadata to /tmp/tmpl0ofr4vq.json [2021-11-29 07:02:28,10] [info] Workflow polling stopped [2021-11-29 07:02:28,12] [info] 0 workflows released by cromid-e9c15e1 [2021-11-29 07:02:28,12] [info] Shutting down WorkflowStoreActor - Timeout = 5 seconds [2021-11-29 07:02:28,12] [info] Shutting down WorkflowLogCopyRouter - Timeout = 5 seconds [2021-11-29 07:02:28,13] [info] Shutting down JobExecutionTokenDispenser - Timeout = 5 seconds [2021-11-29 07:02:28,13] [info] Aborting all running workflows. [2021-11-29 07:02:28,13] [info] JobExecutionTokenDispenser stopped [2021-11-29 07:02:28,13] [info] WorkflowStoreActor stopped [2021-11-29 07:02:28,13] [info] WorkflowLogCopyRouter stopped [2021-11-29 07:02:28,13] [info] Shutting down WorkflowManagerActor - Timeout = 3600 seconds [2021-11-29 07:02:28,13] [info] WorkflowManagerActor All workflows finished [2021-11-29 07:02:28,13] [info] WorkflowManagerActor stopped [2021-11-29 07:02:28,40] [info] Connection pools shut down [2021-11-29 07:02:28,41] [info] Shutting down SubWorkflowStoreActor - Timeout = 1800 seconds [2021-11-29 07:02:28,41] [info] Shutting down JobStoreActor - Timeout = 1800 seconds [2021-11-29 07:02:28,41] [info] Shutting down CallCacheWriteActor - Timeout = 1800 seconds [2021-11-29 07:02:28,41] [info] Shutting down ServiceRegistryActor - Timeout = 1800 seconds [2021-11-29 07:02:28,41] [info] SubWorkflowStoreActor stopped [2021-11-29 07:02:28,41] [info] Shutting down DockerHashActor - Timeout = 1800 seconds [2021-11-29 07:02:28,41] [info] Shutting down IoProxy - Timeout = 1800 seconds [2021-11-29 07:02:28,41] [info] JobStoreActor stopped [2021-11-29 07:02:28,41] [info] CallCacheWriteActor Shutting down: 0 queued messages to process [2021-11-29 07:02:28,41] [info] CallCacheWriteActor stopped [2021-11-29 07:02:28,41] [info] WriteMetadataActor Shutting down: 0 queued messages to process [2021-11-29 07:02:28,41] [info] KvWriteActor Shutting down: 0 queued messages to process [2021-11-29 07:02:28,42] [info] IoProxy stopped [2021-11-29 07:02:28,42] [info] ServiceRegistryActor stopped [2021-11-29 07:02:28,42] [info] DockerHashActor stopped [2021-11-29 07:02:28,45] [info] Database closed [2021-11-29 07:02:28,45] [info] Stream materializer shut down [2021-11-29 07:02:28,47] [info] WDL HTTP import resolver closed [2021-11-29 07:02:28,50] [info] Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false [2021-11-29 07:02:28,50] [info] Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false [2021-11-29 07:02:28,50] [info] Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false [2021-11-29 07:02:28,50] [info] Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false Workflow 3547ffb0-3397-4423-b6b9-18edb10fe366 transitioned to state Failed
It looks like this error is caused by failure of internet connection. But the HPC compute nodes in use don't have internet access. Could you help me with that?
Thank you, Jianbiao Li
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NCIP/ctat-mutations/issues/103, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABZRKX5PTPBFRQQGMD6WWN3UOMZV5ANCNFSM5I6P6I2A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
--
Brian J. Haas The Broad Institute http://broadinstitute.org/~bhaas http://broad.mit.edu/~bhaas
jianbiaoli
taot2007
Hi,
I tried to run CTAT-mutations using singularity. My command is:
/path1/singularity exec --bind /path2/ctat_genome_lib_build_dir,/path4/ /path3/ctat_mutations.v3.2.0.simg /usr/local/src/ctat-mutations/ctat_mutations --genome_lib_dir /path2/ctat_genome_lib_build_dir --cpu 4 --boosting_method none --sample_id RHN22 --bam /path4/RHN22.star.Aligned.sortedByCoord.out.bam -O /path4/mutationsThen I got the error in the log file:
It looks like this error is caused by failure of internet connection. But the HPC compute nodes in use don't have internet access. Could you help me with that?
Thank you, Jianbiao Li