search

ENCODE-DCC / chip-seq-pipeline2

ENCODE ChIP-seq pipeline
MIT License
234 stars 123 forks source link

Error in Alignment process : Call input and runtime attributes evaluation failed for align: Failed to evaluate input 'tmp_fastqs' (reason 1 of 1): transpose requires all collections have the same size #290

Closed cafri1105 closed 1 year ago

cafri1105 commented 1 year ago

Describe the bug

Hi,

Thanks to you, I've been using the chip-seq-pipeline2 workflow since last year until earlier this year which was very helpful.

However, a few days ago, when I ran the new data on the same command as earlier this year, the following error appeared repeatedly.

Failed to evaluate input 'tmp_fastqs' (reason 1 of 1): transpose requires all collections have the same size

cromwell.engine.workflow.lifecycle.execution.job.preparation.JobPreparationActor$$anonfun$1$$anon$1: Call input and runtime attributes evaluation failed for align:
Failed to evaluate input 'tmp_fastqs' (reason 1 of 1): transpose requires all collections have the same size

cromwell.engine.workflow.lifecycle.execution.job.preparation.JobPreparationActor$$anonfun$1$$anon$1: Call input and runtime attributes evaluation failed for align_ctl:
Failed to evaluate input 'tmp_fastqs' (reason 1 of 1): transpose requires all collections have the same size

cromwell.engine.workflow.lifecycle.execution.job.preparation.JobPreparationActor$$anonfun$1$$anon$1: Call input and runtime attributes evaluation failed for align:
Failed to evaluate input 'tmp_fastqs' (reason 1 of 1): transpose requires all collections have the same size

The same alignment process error continues to occur despite updating the caper and chip-seq-pipeline2 to the latest version by referring to the previous issue. https://github.com/ENCODE-DCC/chip-seq-pipeline2/issues/226#issue-856531893

I checked my fastq file , but there was no problem.

I wonder what caused this error. can you please help?

Thanks!

OS/Platform

Caper configuration file

caper run chip.wdl -i lmna_test.json --singularity --leader-job-name ChIP_TEST

Paste contents of ~/.caper/default.conf.

backend=local

# Local directory for localized files and Cromwell's intermediate files.
# If not defined then Caper will make .caper_tmp/ on CWD or `local-out-dir`.
# /tmp is not recommended since Caper store localized data files here.
local-loc-dir=

cromwell=/home/hj/.caper/cromwell_jar/cromwell-82.jar
womtool=/home/hj/.caper/womtool_jar/womtool-82.jar

Input JSON file

{
    "chip.title" : "LMNA ab TEST",
    "chip.description" : "LMNA AB TEST chip data",

    "chip.pipeline_type" : "histone",
    "chip.aligner" : "bowtie2",
    "chip.align_only" : false,
    "chip.true_rep_only" : false,

    "chip.genome_tsv" : "/mnt/NAS2/home/hj/chip-seq-pipeline2/hg19.tsv",

    "chip.paired_end" : true,
    "chip.ctl_paired_end" : true,
    "chip.always_use_pooled_ctl" : true,
    "chip.peak_caller" : "macs2",
    "chip.pval_thresh" : 0.01,
    "chip.fdr_thresh" : 0.01,
    "chip.idr_thresh" : 0.05,
    "chip.enable_jsd" : true,
    "chip.enable_gc_bias" : true,
    "chip.enable_count_signal_track" : true,

    "chip.fastqs_rep1_R1" : ["/mnt/NAS2/home/hj/LMNA_ChIP_TEST/CM_LAab_1.fastq.gz"],
    "chip.fastqs_rep1_R2" : ["/mnt/NAS2/home/hj/LMNA_ChIP_TEST/CM_LAab_2.fastq.gz"],

    "chip.ctl_fastqs_rep1_R1" : ["/mnt/NAS2/home/hj/LMNA_ChIP_TEST/CM_input_1.fastq.gz"],
    "chip.ctl_fastqs_rep1_R2" : ["/mnt/NAS2/home/hj/LMNA_ChIP_TEST/CM_input_2.fastq.gz"]
}

Troubleshooting result

2022-12-26 18:37:28,554  IoFO o- Running with database db.url = jdbc:hsqldb:mem:2332d49b-cffe-4946-9356-ad5653ee3016;shutdown=false;hsqldb.tx=mvcc
ut" 640L, 65209B                                                                                                      640,64        Bot
2022-12-26 20:32:23,077 cromwell-system-akka.dispatchers.backend-dispatcher-1526 INFO  - BackgroundConfigAsyncJobExecutionActor [UUID(faf8a76e)chip.align_R1:1:1]: Status change from WaitingForReturnCode to Done
2022-12-26 20:32:51,042 cromwell-system-akka.dispatchers.backend-dispatcher-1526 INFO  - BackgroundConfigAsyncJobExecutionActor [UUID(faf8a76e)chip.align_R1:0:1]: Status change from WaitingForReturnCode to Done
2022-12-26 20:32:55,845 cromwell-system-akka.dispatchers.engine-dispatcher-31 INFO  - WorkflowManagerActor: Workflow faf8a76e-54b4-45ec-87a5-972b2408e03c failed (during ExecutingWorkflowState): cromwell.engine.workflow.lifecycle.execution.job.preparation.JobPreparationActor$$anonfun$1$$anon$1: Call input and runtime attributes evaluation failed for align_ctl:
Failed to evaluate input 'tmp_fastqs' (reason 1 of 1): transpose requires all collections have the same size

cromwell.engine.workflow.lifecycle.execution.job.preparation.JobPreparationActor$$anonfun$1$$anon$1: Call input and runtime attributes evaluation failed for align:
Failed to evaluate input 'tmp_fastqs' (reason 1 of 1): transpose requires all collections have the same size

cromwell.engine.workflow.lifecycle.execution.job.preparation.JobPreparationActor$$anonfun$1$$anon$1: Call input and runtime attributes evaluation failed for align_ctl:
Failed to evaluate input 'tmp_fastqs' (reason 1 of 1): transpose requires all collections have the same size

cromwell.engine.workflow.lifecycle.execution.job.preparation.JobPreparationActor$$anonfun$1$$anon$1: Call input and runtime attributes evaluation failed for align:
Failed to evaluate input 'tmp_fastqs' (reason 1 of 1): transpose requires all collections have the same size

2022-12-26 20:34:13,871 cromwell-system-akka.dispatchers.engine-dispatcher-20 INFO  - WorkflowManagerActor: Workflow actor for faf8a76e-54b4-45ec-87a5-972b2408e03c completed with status 'Failed'. The workflow will be removed from the workflow store.
2022-12-26 20:34:14,538 cromwell-system-akka.dispatchers.engine-dispatcher-9 INFO  - SingleWorkflowRunnerActor workflow finished with status 'Failed'.
2022-12-26 20:34:19,195 cromwell-system-akka.dispatchers.engine-dispatcher-9 INFO  - SingleWorkflowRunnerActor writing metadata to /mnt/NAS2/home/hj/chip-seq-pipeline2/.caper_tmp/chip/20221226_183721_969493/metadata.json
2022-12-26 20:34:19,417  INFO  - Workflow polling stopped
2022-12-26 20:34:19,422  INFO  - 0 workflows released by cromid-497e9e3
2022-12-26 20:34:19,424  INFO  - Shutting down WorkflowStoreActor - Timeout = 5 seconds
2022-12-26 20:34:19,427  INFO  - Shutting down WorkflowLogCopyRouter - Timeout = 5 seconds
2022-12-26 20:34:19,429  INFO  - Shutting down JobExecutionTokenDispenser - Timeout = 5 seconds
2022-12-26 20:34:19,431  INFO  - WorkflowLogCopyRouter stopped
2022-12-26 20:34:19,431  INFO  - JobExecutionTokenDispenser stopped
2022-12-26 20:34:19,432 cromwell-system-akka.dispatchers.engine-dispatcher-9 INFO  - Aborting all running workflows.
2022-12-26 20:34:19,434  INFO  - WorkflowStoreActor stopped
2022-12-26 20:34:19,434  INFO  - Shutting down WorkflowManagerActor - Timeout = 3600 seconds
2022-12-26 20:34:19,434 cromwell-system-akka.dispatchers.engine-dispatcher-21 INFO  - WorkflowManagerActor: All workflows finished
2022-12-26 20:34:19,434  INFO  - WorkflowManagerActor stopped
2022-12-26 20:34:19,561  INFO  - Connection pools shut down
2022-12-26 20:34:19,562  INFO  - Shutting down SubWorkflowStoreActor - Timeout = 1800 seconds
2022-12-26 20:34:19,562  INFO  - Shutting down JobStoreActor - Timeout = 1800 seconds
2022-12-26 20:34:19,562  INFO  - Shutting down CallCacheWriteActor - Timeout = 1800 seconds
2022-12-26 20:34:19,562  INFO  - Shutting down ServiceRegistryActor - Timeout = 1800 seconds
2022-12-26 20:34:19,562  INFO  - Shutting down DockerHashActor - Timeout = 1800 seconds
2022-12-26 20:34:19,562  INFO  - Shutting down IoProxy - Timeout = 1800 seconds
2022-12-26 20:34:19,564  INFO  - CallCacheWriteActor Shutting down: 0 queued messages to process
2022-12-26 20:34:19,568  INFO  - SubWorkflowStoreActor stopped
2022-12-26 20:34:19,568  INFO  - JobStoreActor stopped
2022-12-26 20:34:19,568  INFO  - CallCacheWriteActor stopped
2022-12-26 20:34:19,568  INFO  - IoProxy stopped
2022-12-26 20:34:19,574  INFO  - DockerHashActor stopped
2022-12-26 20:34:19,575  INFO  - Shutting down connection pool: curAllocated=1 idleQueues.size=1 waitQueue.size=0 maxWaitQueueLimit=256 closed=false
2022-12-26 20:34:19,576  INFO  - Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false
2022-12-26 20:34:19,576  INFO  - Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false
2022-12-26 20:34:19,577  INFO  - WriteMetadataActor Shutting down: 0 queued messages to process
2022-12-26 20:34:19,577  INFO  - KvWriteActor Shutting down: 0 queued messages to process
2022-12-26 20:34:19,582  INFO  - ServiceRegistryActor stopped
2022-12-26 20:34:19,585  INFO  - Database closed
2022-12-26 20:34:19,585  INFO  - Stream materializer shut down
2022-12-26 20:34:19,587  INFO  - WDL HTTP import resolver closed
cafri1105 commented 1 year ago

I found a problem with my JSON file. I solved it. Sorry

olechnwin commented 7 months ago

@cafri1105,

Can you please share what problem you had with your JSON file? I had the same error and I can't find the problem with my JSON file.