search

nf-core / raredisease

Call and score variants from WGS/WES of rare disease patients.
https://nf-co.re/raredisease
MIT License
84 stars 34 forks source link

Join error ME_INDEX_SPLIT_ALIGNMENT #542

Closed fa2k closed 4 months ago

fa2k commented 5 months ago

Description of the bug

I'm getting an error when trying to run the pipeline. The test profile works fine, so it's very likely that I'm doing something wrong, but I can't figure it out. I have not been able to run 2.0.1 yet, apart from the test profile, so this issue is a part of setting it up.

I have disable all the annotation to troubleshoot the basic pipeline.

Command used and terminal output

export NFX_SINGULARITY_CACHEDIR=../raredisease-configs/nf-core-raredisease-dev/singularity-images

~/bin/nextflow run \
    -c ../raredisease-configs/medGenConfigs/process-overrides.conf \
    -c ../raredisease-configs/medGenConfigs/loki-settings.conf \
    ../raredisease-configs/nf-core-raredisease_2.0.1/2_0_1 \
    -params-file ../raredisease-configs/medGenConfigs/grch38-params-simpletest.yaml \
    --input samples.csv \
    --max_memory '750.GB' \
    --max_cpus 64 \
    -profile singularity \
    -resume

[...]

[7e/e7be56] NFCORE_RAREDISEASE:RAREDISEASE:CALL_MOBILE_ELEMENTS:ME_SPLIT_ALIGNMENT (NA12878)                [100%] 25 of 25, cached: 25 ✔
[a8/b58d57] NFCORE_RAREDISEASE:RAREDISEASE:CALL_MOBILE_ELEMENTS:ME_INDEX_SPLIT_ALIGNMENT (NA12878)          [100%] 25 of 25, cached: 25 ✔
Plus 64 more processes waiting for tasks…
Join mismatch for the following entries:
- key=[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr13, nr_of_intervals:25] values=[/data0/paalmbj/na12878_med_pipeline/work/83/20f04adedf5dc4bb5ee4b6b85e262c/NA12878_chr13.bam, /data0/paalmbj/na12878_med_pipeline/work/d0/156dd6f94e30b5c404ec292aad5b8c/NA12878_chr13.bam.bai]
- key=[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chrM, nr_of_intervals:25] values=[/data0/paalmbj/na12878_med_pipeline/work/7e/e7be56a5309cfbe71747b07d6d59c0/NA12878_chrM.bam, /data0/paalmbj/na12878_med_pipeline/work/8e/d6cb493ec2c71d4e88d1cc584d7458/NA12878_chrM.bam.bai]
- key=[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr12, nr_of_intervals:25] values=[/data0/paalmbj/na12878_med_pipeline/work/e2/466c5d4f4e622561dc3ad47208f3c1/NA12878_chr12.bam, /data0/paalmbj/na12878_med_pipeline/work/38/eb7333fa512f82a8b8964a7deb3e20/NA12878_chr12.bam.bai]
- key=[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr19, nr_of_intervals:25] values=[/data0/paalmbj/na12878_med_pipeline/work/fb/ba1a55cf76caa3c4adfece9eddf42b/NA12878_chr19.bam, /data0/paalmbj/na12878_med_pipeline/work/30/2bd2d1df54a4c8557175b9f659c5fc/NA12878_chr19.bam.bai]
- key=[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr18, nr_of_intervals:25] values=[/data0/paalmbj/na12878_med_pipeline/work/44/b4338102a0fd64309fe332f96ab3cd/NA12878_chr18.bam, /data0/paalmbj/na12878_med_pipeline/work/d5/c7ed0a7b253b252beae8772e77ab19/NA12878_chr18.bam.bai]
- key=[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chrY, nr_of_intervals:25] values=[/data0/paalmbj/na12878_med_pipeline/work/d1/db6ab63d03afe53796fe2b003bc206/NA12878_chrY.bam, /data0/paalmbj/na12878_med_pipeline/work/8a/5dbcc7733844da2f1c14305379b5f6/NA12878_chrY.bam.bai]
- key=[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr15, nr_of_intervals:25] values=[/data0/paalmbj/na12878_med_pipeline/work/bf/02f45faa6ed457bbc8d2e275ffdd76/NA12878_chr15.bam, /data0/paalmbj/na12878_med_pipeline/work/9e/d6f461a2bde8b3e1dbc52858be8a5a/NA12878_chr15.bam.bai]
- key=[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr14, nr_of_intervals:25] values=[/data0/paalmbj/na12878_med_pipeline/work/ea/ed99f6a5d7246570ca3f63dd8c5ad0/NA12878_chr14.bam, /data0/paalmbj/na12878_med_pipeline/work/22/e041972bb54b4308433e6aff12470b/NA12878_chr14.bam.bai]
- key=[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr17, nr_of_intervals:25] values=[/data0/paalmbj/na12878_med_pipeline/work/c9/e72a8e390b16aa14f96bbc79dc5b8d/NA12878_chr17.bam, /data0/paalmbj/na12878_med_pipeline/work/db/c164f60348dc47938708c66269b914/NA12878_chr17.bam.bai]
- key=[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr16, nr_of_intervals:25] values=[/data0/paalmbj/na12878_med_pipeline/work/1b/e647d426c15f2cb7d490d17442b519/NA12878_chr16.bam, /data0/paalmbj/na12878_med_pipeline/work/ee/13d7e445aa075a5a3135bfb1e3ed9a/NA12878_chr16.bam.bai]
(more omitted)

WARN: Killing running tasks (7)

Relevant files

nextflow.log samples.csv grch38-params-simpletest.yaml.txt

System information

OS: RHEL 9 Nextflow version 24.03.0-edge build 5908 Pipeline: 2.0.1 or "patch" branch

ramprasadn commented 4 months ago

Could you share

  1. The contents of your fasta index file @fa2k? Log file says it should be at /data0/paalmbj/na12878_med_pipeline/work/9c/96b8c80f1149080deb2195116282e4
  2. Whatever's in this channel here https://github.com/nf-core/raredisease/blob/1489b71cc0e15ade7e98a834df0170b650aa932c/subworkflows/local/call_mobile_elements.nf#L44?
fa2k commented 4 months ago

Thanks for looking at it @ramprasadn . I've pasted the fasta index content below. The genome is the "NCBI" GRCh38 genome from iGenomes. As for the channel contents - Here's the output of ch_genome_bam_bai_interval.view():

[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr1, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr2, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr3, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr4, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr5, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr6, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr7, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr8, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr9, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr10, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr11, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr12, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr13, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr14, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr15, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr16, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr17, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr18, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr19, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr20, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr21, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chr22, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chrX, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chrY, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]
[[id:NA12878, sample:NA12878, lane:1, sex:2, phenotype:2, paternal:0, maternal:0, case_id:NA12878, num_lanes:1, read_group:'@RG\tID:NA12878\tPL:illumina\tSM:NA12878', single_end:false, interval:chrM, nr_of_intervals:25], /data0/paalmbj/na12878_med_pipeline/work/e5/769788e840013284335c41ecac2ea1/NA12878_sorted_md.bam, /data0/paalmbj/na12878_med_pipeline/work/06/fb8da8e9a365cd2abae89737077011/NA12878_sorted_md.bam.bai]

Genome FASTA index:

$ cat /data0/paalmbj/na12878_med_pipeline/work/9c/96b8c80f1149080deb2195116282e4/genome.fa.fai 
chr1    248956422   112 70  71
chr2    242193529   252513167   70  71
chr3    198295559   498166716   70  71
chr4    190214555   699295181   70  71
chr5    181538259   892227221   70  71
chr6    170805979   1076358996  70  71
chr7    159345973   1249605173  70  71
chr8    145138636   1411227630  70  71
chr9    138394717   1558439788  70  71
chr10   133797422   1698811686  70  71
chr11   135086622   1834520613  70  71
chr12   133275309   1971537157  70  71
chr13   114364328   2106716512  70  71
chr14   107043718   2222714743  70  71
chr15   101991189   2331287770  70  71
chr16   90338345    2434736088  70  71
chr17   83257441    2526365093  70  71
chr18   80373285    2610812039  70  71
chr19   58617616    2692333639  70  71
chr20   64444167    2751788762  70  71
chr21   46709983    2817153685  70  71
chr22   50818468    2864531079  70  71
chrX    156040895   2916075638  70  71
chrY    57227415    3074345836  70  71
chrM    16569   3132390908  70  71
chr1_KI270706v1_random  175055  3132407851  70  71
chr1_KI270707v1_random  32032   3132585543  70  71
chr1_KI270708v1_random  127682  3132618170  70  71
chr1_KI270709v1_random  66860   3132747813  70  71
chr1_KI270710v1_random  40176   3132815765  70  71
chr1_KI270711v1_random  42210   3132856651  70  71
chr1_KI270712v1_random  176043  3132899601  70  71
chr1_KI270713v1_random  40745   3133078295  70  71
chr1_KI270714v1_random  41717   3133119759  70  71
chr2_KI270715v1_random  161471  3133162209  70  71
chr2_KI270716v1_random  153799  3133326124  70  71
chr3_GL000221v1_random  155397  3133482258  70  71
chr4_GL000008v2_random  209709  3133640012  70  71
chr5_GL000208v1_random  92689   3133852853  70  71
chr9_KI270717v1_random  40062   3133947003  70  71
chr9_KI270718v1_random  38054   3133987774  70  71
chr9_KI270719v1_random  176845  3134026509  70  71
chr9_KI270720v1_random  39050   3134206017  70  71
chr11_KI270721v1_random 100316  3134245764  70  71
chr14_GL000009v2_random 201709  3134347653  70  71
chr14_GL000225v1_random 211173  3134552383  70  71
chr14_KI270722v1_random 194050  3134766712  70  71
chr14_GL000194v1_random 191469  3134963674  70  71
chr14_KI270723v1_random 38115   3135158017  70  71
chr14_KI270724v1_random 39555   3135196815  70  71
chr14_KI270725v1_random 172810  3135237075  70  71
chr14_KI270726v1_random 43739   3135412492  70  71
chr15_KI270727v1_random 448248  3135456995  70  71
chr16_KI270728v1_random 1872759 3135911787  70  71
chr17_GL000205v2_random 185591  3137811439  70  71
chr17_KI270729v1_random 280839  3137999821  70  71
chr17_KI270730v1_random 112551  3138284811  70  71
chr22_KI270731v1_random 150754  3138399109  70  71
chr22_KI270732v1_random 41543   3138552155  70  71
chr22_KI270733v1_random 179772  3138594431  70  71
chr22_KI270734v1_random 165050  3138776911  70  71
chr22_KI270735v1_random 42811   3138944457  70  71
chr22_KI270736v1_random 181920  3138988019  70  71
chr22_KI270737v1_random 103838  3139172677  70  71
chr22_KI270738v1_random 99375   3139278137  70  71
chr22_KI270739v1_random 73985   3139379070  70  71
chrY_KI270740v1_random  37240   3139454248  70  71
chrUn_KI270302v1    2274    3139492137  70  71
chrUn_KI270304v1    2165    3139494561  70  71
chrUn_KI270303v1    1942    3139496874  70  71
chrUn_KI270305v1    1472    3139498961  70  71
chrUn_KI270322v1    21476   3139500573  70  71
chrUn_KI270320v1    4416    3139522473  70  71
chrUn_KI270310v1    1201    3139527070  70  71
chrUn_KI270316v1    1444    3139528406  70  71
chrUn_KI270315v1    2276    3139529988  70  71
chrUn_KI270312v1    998 3139532413  70  71
chrUn_KI270311v1    12399   3139533544  70  71
chrUn_KI270317v1    37690   3139546239  70  71
chrUn_KI270412v1    1179    3139584585  70  71
chrUn_KI270411v1    2646    3139585898  70  71
chrUn_KI270414v1    2489    3139588699  70  71
chrUn_KI270419v1    1029    3139591341  70  71
chrUn_KI270418v1    2145    3139592502  70  71
chrUn_KI270420v1    2321    3139594795  70  71
chrUn_KI270424v1    2140    3139597267  70  71
chrUn_KI270417v1    2043    3139599555  70  71
chrUn_KI270422v1    1445    3139601745  70  71
chrUn_KI270423v1    981 3139603327  70  71
chrUn_KI270425v1    1884    3139604440  70  71
chrUn_KI270429v1    1361    3139606468  70  71
chrUn_KI270442v1    392061  3139607968  70  71
chrUn_KI270466v1    1233    3140005747  70  71
chrUn_KI270465v1    1774    3140007115  70  71
chrUn_KI270467v1    3920    3140009032  70  71
chrUn_KI270435v1    92983   3140013126  70  71
chrUn_KI270438v1    112505  3140107557  70  71
chrUn_KI270468v1    4055    3140221787  70  71
chrUn_KI270510v1    2415    3140226017  70  71
chrUn_KI270509v1    2318    3140228584  70  71
chrUn_KI270518v1    2186    3140231053  70  71
chrUn_KI270508v1    1951    3140233388  70  71
chrUn_KI270516v1    1300    3140235484  70  71
chrUn_KI270512v1    22689   3140236921  70  71
chrUn_KI270519v1    138126  3140260054  70  71
chrUn_KI270522v1    5674    3140400271  70  71
chrUn_KI270511v1    8127    3140406144  70  71
chrUn_KI270515v1    6361    3140414505  70  71
chrUn_KI270507v1    5353    3140421074  70  71
chrUn_KI270517v1    3253    3140426621  70  71
chrUn_KI270529v1    1899    3140430038  70  71
chrUn_KI270528v1    2983    3140432082  70  71
chrUn_KI270530v1    2168    3140435225  70  71
chrUn_KI270539v1    993 3140437540  70  71
chrUn_KI270538v1    91309   3140438666  70  71
chrUn_KI270544v1    1202    3140531397  70  71
chrUn_KI270548v1    1599    3140532734  70  71
chrUn_KI270583v1    1400    3140534473  70  71
chrUn_KI270587v1    2969    3140536010  70  71
chrUn_KI270580v1    1553    3140539139  70  71
chrUn_KI270581v1    7046    3140540832  70  71
chrUn_KI270579v1    31033   3140548097  70  71
chrUn_KI270589v1    44474   3140579692  70  71
chrUn_KI270590v1    4685    3140624919  70  71
chrUn_KI270584v1    4513    3140629788  70  71
chrUn_KI270582v1    6504    3140634483  70  71
chrUn_KI270588v1    6158    3140641197  70  71
chrUn_KI270593v1    3041    3140647560  70  71
chrUn_KI270591v1    5796    3140650762  70  71
chrUn_KI270330v1    1652    3140656758  70  71
chrUn_KI270329v1    1040    3140658551  70  71
chrUn_KI270334v1    1368    3140659723  70  71
chrUn_KI270333v1    2699    3140661228  70  71
chrUn_KI270335v1    1048    3140664083  70  71
chrUn_KI270338v1    1428    3140665263  70  71
chrUn_KI270340v1    1428    3140666829  70  71
chrUn_KI270336v1    1026    3140668395  70  71
chrUn_KI270337v1    1121    3140669553  70  71
chrUn_KI270363v1    1803    3140670808  70  71
chrUn_KI270364v1    2855    3140672754  70  71
chrUn_KI270362v1    3530    3140675767  70  71
chrUn_KI270366v1    8320    3140679465  70  71
chrUn_KI270378v1    1048    3140688021  70  71
chrUn_KI270379v1    1045    3140689201  70  71
chrUn_KI270389v1    1298    3140690378  70  71
chrUn_KI270390v1    2387    3140691812  70  71
chrUn_KI270387v1    1537    3140694351  70  71
chrUn_KI270395v1    1143    3140696027  70  71
chrUn_KI270396v1    1880    3140697304  70  71
chrUn_KI270388v1    1216    3140699328  70  71
chrUn_KI270394v1    970 3140700678  70  71
chrUn_KI270386v1    1788    3140701779  70  71
chrUn_KI270391v1    1484    3140703710  70  71
chrUn_KI270383v1    1750    3140705333  70  71
chrUn_KI270393v1    1308    3140707225  70  71
chrUn_KI270384v1    1658    3140708669  70  71
chrUn_KI270392v1    971 3140710467  70  71
chrUn_KI270381v1    1930    3140711569  70  71
chrUn_KI270385v1    990 3140713643  70  71
chrUn_KI270382v1    4215    3140714765  70  71
chrUn_KI270376v1    1136    3140719158  70  71
chrUn_KI270374v1    2656    3140720428  70  71
chrUn_KI270372v1    1650    3140723239  70  71
chrUn_KI270373v1    1451    3140725030  70  71
chrUn_KI270375v1    2378    3140726619  70  71
chrUn_KI270371v1    2805    3140729148  70  71
chrUn_KI270448v1    7992    3140732111  70  71
chrUn_KI270521v1    7642    3140740335  70  71
chrUn_GL000195v1    182896  3140748206  70  71
chrUn_GL000219v1    179198  3140933834  70  71
chrUn_GL000220v1    161802  3141115711  70  71
chrUn_GL000224v1    179693  3141279944  70  71
chrUn_KI270741v1    157432  3141462324  70  71
chrUn_GL000226v1    15008   3141622124  70  71
chrUn_GL000213v1    164239  3141637466  70  71
chrUn_KI270743v1    210658  3141804171  70  71
chrUn_KI270744v1    168472  3142017958  70  71
chrUn_KI270745v1    41891   3142188955  70  71
chrUn_KI270746v1    66486   3142231563  70  71
chrUn_KI270747v1    198735  3142299118  70  71
chrUn_KI270748v1    93321   3142500811  70  71
chrUn_KI270749v1    158759  3142595585  70  71
chrUn_KI270750v1    148850  3142756731  70  71
chrUn_KI270751v1    150742  3142907827  70  71
chrUn_KI270752v1    27745   3143060841  70  71
chrUn_KI270753v1    62944   3143089101  70  71
chrUn_KI270754v1    40191   3143153063  70  71
chrUn_KI270755v1    36723   3143193947  70  71
chrUn_KI270756v1    79590   3143231313  70  71
chrUn_KI270757v1    71251   3143312158  70  71
chrUn_GL000214v1    137718  3143384546  70  71
chrUn_KI270742v1    186739  3143524351  70  71
chrUn_GL000216v2    176608  3143713877  70  71
chrUn_GL000218v1    161147  3143893127  70  71
chrEBV  171823  3144056708  70  71
ramprasadn commented 4 months ago

@fa2k Can you try out the latest dev and let me know if that works?

fa2k commented 4 months ago

Testing now- will let you know, but it takes some time

fa2k commented 4 months ago

I still got the same issue when using the dev branch and my usual options. But I should probably try to reduce my custom options, disable features and also change to a standard reference like "--genome GRCh38". (Usually I specify --fasta, --fai, etc.) I'll let you know how this goes within the next few days.

ramprasadn commented 4 months ago

Great! Sometimes these errors when joining data can be misleading, and there may be another issue at play (such as a process not running or empty inputs causing errors during the join operation). I suggest making a local copy of the repository, removing all calls downstream of ALIGN, and gradually working through the code to identify the source of the error. If you need assistance, feel free to reach out. We can schedule a call to troubleshoot and resolve the issue together.

fa2k commented 4 months ago

That's a great tip. I got it to run by commenting out a lot of the workflow. Now I will progressively add back stuff to see where the error comes from. Also thanks for offering to assist remotely, I'll let you know if I get stuck / have many questions.

fa2k commented 4 months ago

The error comes up if I include CALL_MOBILE_ELEMENTS, and I can run the entire workflow if I leave out CALL_MOBILE_ELEMENTS.

The ch_me_reference_split channel is empty because I haven't specified the parameter --mobile_element_references.

If I specify --mobile_element_references as the one from the test profile, the full workflow works! (I'm sure the ME will produce nonsensical results, as the test file is for grch37).

I think either mobile_element_references should be marked as a required parameter, or maybe the subworkflow can be skipped if mobile_element_references is not given.

ramprasadn commented 4 months ago

Awesome! Yeah, I will make changes in a PR to rectify this! Thanks @fa2k

ramprasadn commented 4 months ago

Fixed in #556