search

icbi-lab / nextNEOpi

nextNEOpi: a comprehensive pipeline for computational neoantigen prediction
Other
65 stars 23 forks source link

Process `get_vhla (test)` terminated with an error exit status (1) #67

Closed rashidma closed 2 months ago

rashidma commented 7 months ago

After I fixed several issues with your help, the new issue popped up. Below is the error:

Error_ executing process > 'get_vhla (test)'

Caused by:
  Process `get_vhla (test)` terminated with an error exit status (1)

Command executed:

  # merging script
  HLA_parser.py \
      --opti_out test_optitype_result.tsv \
      --hlahd_out test_tumor_DNA_final.result.txt \
      --opti_out_RNA test_optitype_RNA_result.tsv \
       \
       \
      --ref_hlas /mnt/e/Mamoon/software/nextNEOpi-master/assets/pVACseqAlleles.txt \
      > ./test_hlas.txt

Command exit status:
  1

Command output:
  (empty)

Command error:
  Traceback (most recent call last):
    File "/mnt/e/Mamoon/software/nextNEOpi-master/bin/HLA_parser.py", line 423, in <module>
      parse_hlahd(hlahd_dna_result, "classII", hlaII_array)
    File "/mnt/e/Mamoon/software/nextNEOpi-master/bin/HLA_parser.py", line 73, in parse_hlahd
      if "HLA-" in line[1]:
  IndexError: list index out of range

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`  __

Please help to troubleshoot. It looks like "HLA_parser.py" is having some bug in indexing!

Thanks a lot

riederd commented 7 months ago

can you send the contents of the work folder in which the get_vhla failed as tar.gz

rashidma commented 7 months ago

Dear Rieder, The failed work dir is below: /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/d4/4d565442ef4ad7f8cb1139bc838109

Please find attached the entire d4 directory.

Thank you for your time. I have been trying to run nextNEOpi on demo data since long time. Unfortunately, I am still suffering due to issues. Please help me to an extent so that I can run this on the demo successfully.

Please let me know. Best regards Mamoon

On Mon, Dec 18, 2023 at 12:40 PM Dietmar Rieder @.***> wrote:

can you send the contents of the work folder in which the get_vhla failed as tar.gz

— Reply to this email directly, view it on GitHub https://github.com/icbi-lab/nextNEOpi/issues/67#issuecomment-1859946440, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE6KEOFZSVU7IK7X2IWNIDTYKAFPDAVCNFSM6AAAAABAZDJCE6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJZHE2DMNBUGA . You are receiving this because you authored the thread.Message ID: @.***>

--

Mamoon Rashid, PhD (Bioinformatics)

Associate Research Scientist

Department of Biostatistics and Bioinformatics

King Abdullah International Medical Research Center (KAIMRC) Kingdom of Saudi Arabia

riederd commented 7 months ago

Hi, can it be that you forgot to attach?

rashidma commented 7 months ago

I can see attachment in my previous email. d4.tar.gz Please check.

Mamoon

On Mon, 18 Dec 2023 at 6:20 PM Dietmar Rieder @.***> wrote:

Hi, can it be that you forgot to attach?

— Reply to this email directly, view it on GitHub https://github.com/icbi-lab/nextNEOpi/issues/67#issuecomment-1860792511, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE6KEOCZX5SCPLHYLZ3UDADYKBNLFAVCNFSM6AAAAABAZDJCE6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRQG44TENJRGE . You are receiving this because you authored the thread.Message ID: @.***>

riederd commented 7 months ago

There is no attachment in github and also not in the notification e-mail that I got from github. Sorry

rashidma commented 7 months ago

d4.tar.gz

riederd commented 7 months ago

Now it is there, however the symlinks are not followed, can you please create the tar.gz using the following command:

tar -chzf work_dir.tar.gz /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/d4/4d565442ef4ad7f8cb1139bc838109

thanks

rashidma commented 7 months ago

Done. Please see below: work_dir.tar.gz

riederd commented 7 months ago

Hi and thanks,

it seems that HLA-HD did not run successfully and created an unexpected output (test_tumor_DNA_final.result.txt)

can you sent the contents of: /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/88/273dc3f21d5c333657d8e19200b919/test_tumor_DNA/log

and

/mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/88/273dc3f21d5c333657d8e19200b919/.command.err /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/88/273dc3f21d5c333657d8e19200b919/.command.log /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/88/273dc3f21d5c333657d8e19200b919/.command.out

rashidma commented 7 months ago

Okay. Please have the tarballs below: log.tar.gz

command.err.tar.gz command.log.tar.gz command.out.tar.gz

riederd commented 7 months ago

I see that your hlahd installation is missing bowtie2 on your system:

/mnt/e/Mamoon/software/HLAHD/hlahd.1.5.0/bin/hlahd.sh: line 97: bowtie2: command not found

I suggest you install bowtie2 as stated on https://w3.genome.med.kyoto-u.ac.jp/HLA-HD/ Then it should work.

rashidma commented 7 months ago

Hi, I installed "bowtie2" and set the path to "bash_profile". I tested that it is working on command prompt. I also set the path of "hlahd_dir/bin" in my PATH variable.

Having said that, nextNEOpi again failed with the same error.

Error executing process > 'get_vhla (test)'

Caused by:
  Process `get_vhla (test)` terminated with an error exit status (1)

Command executed:

  # merging script
  HLA_parser.py \
      --opti_out test_optitype_result.tsv \
      --hlahd_out test_tumor_DNA_final.result.txt \
      --opti_out_RNA test_optitype_RNA_result.tsv \
       \
       \
      --ref_hlas /mnt/e/Mamoon/software/nextNEOpi-master/assets/pVACseqAlleles.txt \
      > ./test_hlas.txt

Command exit status:
  1

Command output:
  (empty)

Command error:
  Traceback (most recent call last):
    File "/mnt/e/Mamoon/software/nextNEOpi-master/bin/HLA_parser.py", line 423, in <module>
      parse_hlahd(hlahd_dna_result, "classII", hlaII_array)
    File "/mnt/e/Mamoon/software/nextNEOpi-master/bin/HLA_parser.py", line 73, in parse_hlahd
      if "HLA-" in line[1]:
  IndexError: list index out of range

Work dir:
  /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/d9/5783971477e8cda092ee113063e03a

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

I checked "test_tumor_DNA_final.result.txt", again failed. I am surprised with this situation.

Thanks a lot

riederd commented 7 months ago

Can you again send the contents of

/mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/d9/5783971477e8cda092ee113063e03a

As tar.gz with the -chzf option

rashidma commented 7 months ago

d9.tar.gz Thanks

riederd commented 7 months ago

Hi and thanks,

it still seems that HLA-HD did not run successfully and created an unexpected output (test_tumor_DNA_final.result.txt)

can you sent the contents of:

/mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/dd/b93680559f929b4300e69e79c18561/test_tumor_DNA/log and

/mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/dd/b93680559f929b4300e69e79c18561/.command.err /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/dd/b93680559f929b4300e69e79c18561/.command.log /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/dd/b93680559f929b4300e69e79c18561/.command.out

rashidma commented 7 months ago

Here you go. HLAHD is 1.5, do you think I should use version 1.7?

command.err.tar.gz command.out.tar.gz command.log.tar.gz log.tar.gz

rashidma commented 7 months ago

I checked the .command.err file and it looks like bowtie2 is not accessible! But, i installed it at /mnt/e/Mamoon/software/bowtie2-2.5.2-linux-x86_64/bowtie2 and included in the path in bash_profile and bashrc files.

Please comment. Thank you

riederd commented 6 months ago

Can you run hlahd manually, I mean independent of the nextNEOpi? Is it finding bowtie2

rashidma commented 6 months ago

I run hlahd manually using a demo published on their website hlahd-demo.v1.pdf

The command i used is as below : hlahd.sh -t 4 -m 100 -f /mnt/e/Mamoon/software/HLAHD/hlahd.1.5.0/freq_data ./SRR794547_1.fastq ./SRR794547_2.fastq /mnt/e/Mamoon/software/HLAHD/hlahd.1.5.0/HLA_gene.split.txt /mnt/e/Mamoon/software/HLAHD/hlahd.1.5.0/dictionary SRR794547 . The output as below:

HLA-HD version 1.5.0
22082652 reads; of these:
  22082652 (100.00%) were unpaired; of these:
    21913326 (99.23%) aligned 0 times
    16595 (0.08%) aligned exactly 1 time
    152731 (0.69%) aligned >1 times
0.77% overall alignment rate
22082652 reads; of these:
  22082652 (100.00%) were unpaired; of these:
    21910199 (99.22%) aligned 0 times
    16670 (0.08%) aligned exactly 1 time
    155783 (0.71%) aligned >1 times
0.78% overall alignment rate
169326 reads; of these:
  169326 (100.00%) were unpaired; of these:
    162436 (95.93%) aligned 0 times
    2161 (1.28%) aligned exactly 1 time
    4729 (2.79%) aligned >1 times
4.07% overall alignment rate
172453 reads; of these:
  172453 (100.00%) were unpaired; of these:
    166169 (96.36%) aligned 0 times
    2126 (1.23%) aligned exactly 1 time
    4158 (2.41%) aligned >1 times
3.64% overall alignment rate
169326 reads; of these:
  169326 (100.00%) were unpaired; of these:
    162555 (96.00%) aligned 0 times
    2517 (1.49%) aligned exactly 1 time
    4254 (2.51%) aligned >1 times
4.00% overall alignment rate
172453 reads; of these:
  172453 (100.00%) were unpaired; of these:
    166146 (96.34%) aligned 0 times
    2266 (1.31%) aligned exactly 1 time
    4041 (2.34%) aligned >1 times
3.66% overall alignment rate
156014 reads; of these:
  156014 (100.00%) were unpaired; of these:
    149987 (96.14%) aligned 0 times
    620 (0.40%) aligned exactly 1 time
    5407 (3.47%) aligned >1 times
3.86% overall alignment rate
160131 reads; of these:
  160131 (100.00%) were unpaired; of these:
    152865 (95.46%) aligned 0 times
    742 (0.46%) aligned exactly 1 time
    6524 (4.07%) aligned >1 times
4.54% overall alignment rate
156014 reads; of these:
  156014 (100.00%) were unpaired; of these:
    2764 (1.77%) aligned 0 times
    16057 (10.29%) aligned exactly 1 time
    137193 (87.94%) aligned >1 times
98.23% overall alignment rate
160131 reads; of these:
  160131 (100.00%) were unpaired; of these:
    3317 (2.07%) aligned 0 times
    16148 (10.08%) aligned exactly 1 time
    140666 (87.84%) aligned >1 times
97.93% overall alignment rate
42871 reads; of these:
  42871 (100.00%) were unpaired; of these:
    41384 (96.53%) aligned 0 times
    205 (0.48%) aligned exactly 1 time
    1282 (2.99%) aligned >1 times
3.47% overall alignment rate
41708 reads; of these:
  41708 (100.00%) were unpaired; of these:
    39940 (95.76%) aligned 0 times
    238 (0.57%) aligned exactly 1 time
    1530 (3.67%) aligned >1 times
4.24% overall alignment rate
169326 reads; of these:
  169326 (100.00%) were unpaired; of these:
    8638 (5.10%) aligned 0 times
    15012 (8.87%) aligned exactly 1 time
    145676 (86.03%) aligned >1 times
94.90% overall alignment rate
172453 reads; of these:
  172453 (100.00%) were unpaired; of these:
    8675 (5.03%) aligned 0 times
    15082 (8.75%) aligned exactly 1 time
    148696 (86.22%) aligned >1 times
94.97% overall alignment rate

Ì also checked the final result SRR794547/result/SRR794547_final.result.txt and as below:

A       HLA-A*31:01:02  HLA-A*02:01:01
B       HLA-B*07:02:01  HLA-B*15:01:01
C       HLA-C*03:03:01  HLA-C*07:02:01
DRB1    HLA-DRB1*15:01:01       HLA-DRB1*04:04:01
DQA1    HLA-DQA1*01:02:01       HLA-DQA1*03:01:01
DQB1    HLA-DQB1*06:02:01       HLA-DQB1*03:02:01
DPA1    HLA-DPA1*01:03:01       -
DPB1    HLA-DPB1*02:01:02       HLA-DPB1*06:01
DMA     HLA-DMA*01:01:01        -
DMB     HLA-DMB*01:01:01        -
DOA     HLA-DOA*01:01:02        HLA-DOA*01:01:01
DOB     HLA-DOB*01:01:01        HLA-DOB*01:04:01
DRA     HLA-DRA*01:02:03        HLA-DRA*01:01:01
DRB2    Not typed       Not typed
DRB3    HLA-DRB3*02:02:01       -
DRB4    HLA-DRB4*01:03:01       HLA-DRB4*01:03:01
DRB5    HLA-DRB5*01:01:01       -
DRB6    HLA-DRB6*02:01  -
DRB7    HLA-DRB7*01:01:01       -
DRB8    HLA-DRB8*01:01  -
DRB9    HLA-DRB9*01:01  -
E       HLA-E*01:01:01  HLA-E*01:03:02
F       HLA-F*01:01:01  HLA-F*01:01:03
G       HLA-G*01:03:01  HLA-G*01:01:01
H       HLA-H*01:01:01  HLA-H*02:04
J       HLA-J*01:01:01  -
K       HLA-K*01:01:01  HLA-K*01:02
L       HLA-L*01:01:01  HLA-L*01:02
V       HLA-V*01:01:01  -

So hlahd is running fine manually outside the nextNEOpi. Thanks

riederd commented 6 months ago

Can you post the .command.run from /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/dd/b93680559f929b4300e69e79c18561/

rashidma commented 6 months ago

command.run.tar.gz thanks

riederd commented 6 months ago

Do you run nextNEOpi on a cluster? If yes, then please make sure that also your compute nodes have bowtie2 installed and located somewhere in your PATH

rashidma commented 6 months ago

Hi, I am not running on a cluster. I am running nextNEOpi on a Windows 11 machine using WSL2 ubuntu installed. This workstation has: 48GB RAM 16 CPUs I did set the PATH variable for bowtie2 in the Ubuntu. Any comment? Thanks for your time

rashidma commented 6 months ago

Hi, I know my troubleshooting has been painful. Do you know any groups where I can interact with people who is running nextNEOpi successfully? Also, I may find someone running this tool on WSL!

Thank you for your time

riederd commented 6 months ago

Hi, we run it regularly at our institute and it works fine on native Linux. Unfortunately we do not use WSL and we have no information about other groups using WSL, so it is difficult for us to help you with WSL.

It seems, however, that somehow the path to bowtie2 is not correctly set when launching the pipeline.

Can you run the following nextflow code, and post the output:

nextflow.enable.dsl = 1

process TEST_PATH {
    debug true

    input:
    output:

    script:
    """
    echo "PATH:"
    echo \$PATH
    echo ""
    """
}

process TEST_BWT2 {
    debug true

    input:
    output:

    script:
    """
    echo "BOWTIE2"
    bowtie2 --version
    echo ""
    """
}

save the code as test.nf and run it as follows:

NXF_VER=22.10.8 nextflow run test.nf

rashidma commented 6 months ago

below is the command:

Analysis/nextNEOpi/demo$ NXF_VER=22.10.8 nextflow run test.nf

below is output:

N E X T F L O W  ~  version 22.10.8
Launching `test.nf` [friendly_caravaggio] DSL1 - revision: 0fa69dde75
executor >  local (2)
[9f/3abc96] process > TEST_PATH [100%] 1 of 1 ✔
[64/74d8f7] process > TEST_BWT2 [  0%] 0 of 1
PATH:
/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2/bin:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2/libnvvp:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/bin:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/libnvvp:/mnt/c/Program Files/Eclipse Adoptium/jdk-11.0.13.8-hotspot/bin:/mnt/c/Program Files/Eclipse Adoptium/jdk-17.0.1.12-hotspot/bin:/mnt/c/windows/system32:/mnt/c/windows:/mnt/c/windows/System32/Wbem:/mnt/c/windows/System32/WindowsPowerShell/v1.0/:/mnt/c/windows/System32/OpenSSH/:/mnt/c/Program Files/NVIDIA Corporation/NVIDIA NvDLISR:/mnt/c/Program Files/MATLAB/R2020b/bin:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.0/extras/CUPTI/lib64:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.0/include:/mnt/c/tools/cuda/bin:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0/:/mnt/c/WINDOWS/System32/OpenSSH/:/mnt/c/Program Files/Git/cmd:/mnt/c/Program Files/NVIDIA Corporation/Nsight Compute 2020.3.0/:/mnt/c/Program Files (xexecutor >  local (2)
[9f/3abc96] process > TEST_PATH [100%] 1 of 1 ✔
[64/74d8f7] process > TEST_BWT2 [100%] 1 of 1 ✔
PATH:
/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2/bin:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2/libnvvp:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/bin:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1/libnvvp:/mnt/c/Program Files/Eclipse Adoptium/jdk-11.0.13.8-hotspot/bin:/mnt/c/Program Files/Eclipse Adoptium/jdk-17.0.1.12-hotspot/bin:/mnt/c/windows/system32:/mnt/c/windows:/mnt/c/windows/System32/Wbem:/mnt/c/windows/System32/WindowsPowerShell/v1.0/:/mnt/c/windows/System32/OpenSSH/:/mnt/c/Program Files/NVIDIA Corporation/NVIDIA NvDLISR:/mnt/c/Program Files/MATLAB/R2020b/bin:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.0/extras/CUPTI/lib64:/mnt/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.0/include:/mnt/c/tools/cuda/bin:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0/:/mnt/c/WINDOWS/System32/OpenSSH/:/mnt/c/Program Files/Git/cmd:/mnt/c/Program Files/NVIDIA Corporation/Nsight Compute 2020.3.0/:/mnt/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0/:/mnt/c/WINDOWS/System32/OpenSSH/:/mnt/c/Users/dralmazroa/AppData/Local/Programs/Julia-1.10.0/bin:/mnt/c/Users/dralmazroa/.julia/packages/VariantVisualization/1yoNl:/snap/bin:/home/mamoon/.dotnet/tools:/mnt/e/Mamoon/software:/home/mamoon/go/bin:/mnt/e/Mamoon/software/bowtie2-2.5.2-linux-x86_64:/mnt/e/Mamoon/software/HLAHD/hlahd.1.5.0/bin

BOWTIE2
/mnt/e/Mamoon/software/bowtie2-2.5.2-linux-x86_64/bowtie2-align-s version 2.5.2
64-bit
Built on 3da406ad167d
Fri Oct 13 23:41:48 UTC 2023
Compiler: gcc version 9.3.1 20200408 (Red Hat 9.3.1-2) (GCC)
Options: -O3 -msse2 -funroll-loops -g3 -g -O2 -fvisibility=hidden -I/hbb_exe_gc_hardened/include -ffunction-sections -fdata-sections -fstack-protector -D_FORTIFY_SOURCE=2 -fPIE -std=c++11 -DPOPCNT_CAPABILITY -DNO_SPINLOCK -DWITH_QUEUELOCK=1 -DWITH_ZSTD
Sizeof {int, long, long long, void*, size_t, off_t}: {4, 8, 8, 8, 8, 8}
riederd commented 6 months ago

Strange that "PATH:" is printed out twice and that the line below it is different. In the first case the bowtie2 path is missing. However, there should be only one output for "PATH:" if you are running exactly the code from above.

Can you edit the nextNEOpi.nf and add echo \$PATH before and after line https://github.com/icbi-lab/nextNEOpi/blob/fe7b21cdc0b97aae38195e5f5ac1b9851674f6b1/nextNEOpi.nf#L5024

Then run the example again and post the .command.log .command.err .command.out from the hlahd process work dir.

rashidma commented 6 months ago

I added "echo \$PATH" as you mentioned. Surprisingly, "get_vhla (test)" is successful this time. I am really surprised. Do you have an explanation for this behavior? The pipeline is still running. I will update you on the status. Many thanks

rashidma commented 6 months ago

Now it broke on "cnvkit (test)" stage. Please see the error below:

Error executing process > 'CNVkit (test)'

Caused by:
  Process `CNVkit (test)` terminated with an error exit status (1)

Command executed:

  # set Agg as backend for matplotlib
  export MATPLOTLIBRC="./matplotlibrc"
  echo "backend : Agg" > $MATPLOTLIBRC

  cnvkit.py \
      batch \
      test_tumor_DNA_aligned_sort_mkdp.bam \
      --normal test_normal_DNA_aligned_sort_mkdp.bam \
      --method hybrid \
      --targets S07604514_Covered.bed \
      --fasta GRCh38.d1.vd1.fa \
      --annotate gencode.v33.primary_assembly.annotation.gtf \
      --access access-5kb.GRCh38.bed \
       \
      -p 16 \
      --output-reference output_reference.cnn \
      --output-dir ./

  cnvkit.py segmetrics \
      -s test_tumor_DNA_aligned_sort_mkdp.cn{s,r} \
      --ci \
      --pi

  cnvkit.py call \
      test_tumor_DNA_aligned_sort_mkdp.cns \
      --filter ci \
      -m clonal \
      --purity 0.55 \
      --sample-sex female \
       \
      -o test_tumor_DNA_aligned_sort_mkdp.call.cns

  cnvkit.py \
      scatter \
      test_tumor_DNA_aligned_sort_mkdp.cnr \
      -s test_tumor_DNA_aligned_sort_mkdp.cns \
      -o test_tumor_DNA_aligned_sort_mkdp_scatter.png

  cnvkit.py \
      diagram \
      test_tumor_DNA_aligned_sort_mkdp.cnr \
      -s test_tumor_DNA_aligned_sort_mkdp.cns \
      --sample-sex female \
       \
      -o test_tumor_DNA_aligned_sort_mkdp_diagram.pdf

  cnvkit.py \
      breaks \
      test_tumor_DNA_aligned_sort_mkdp.cnr test_tumor_DNA_aligned_sort_mkdp.cns \
      -o test_tumor_DNA_aligned_sort_mkdp_breaks.tsv

  cnvkit.py \
      genemetrics \
      test_tumor_DNA_aligned_sort_mkdp.cnr \
      -s test_tumor_DNA_aligned_sort_mkdp.cns \
      --sample-sex female \
       \
      -t 0.2 -m 5 \
      -o test_tumor_DNA_aligned_sort_mkdp_gainloss.tsv

  # run PDF to PNG conversion if mogrify and gs is installed
  mogrify -version > /dev/null 2>&1 && \
  gs -v > /dev/null 2>&1 && \
      mogrify -density 600 -resize 2000 -format png *.pdf

  # clean up
  rm -f $MATPLOTLIBRC

Command exit status:
  1

Command output:
  (empty)

Command error:
  Summary: #bins=280504, #reads=1184722, mean=4.2236, min=0.0, max=2244.1825396825398
  Percent reads in regions: 51.077 (of 2319488 mapped)
  Wrote ./test_tumor_DNA_aligned_sort_mkdp.targetcoverage.cnn with 280504 regions
  Processing reads in test_tumor_DNA_aligned_sort_mkdp.bam
  Time: 2.705 seconds (178542 reads/sec, 15074 bins/sec)
  Summary: #bins=40780, #reads=483025, mean=11.8447, min=0.0, max=10338.79365079365
  Percent reads in regions: 20.825 (of 2319488 mapped)
  Wrote ./test_tumor_DNA_aligned_sort_mkdp.antitargetcoverage.cnn with 40780 regions
  Processing target: test_tumor_DNA_aligned_sort_mkdp
  Keeping 277 of 280504 bins
  WARNING: most bins have no or very low coverage; check that the right BED file was used
  Processing antitarget: test_tumor_DNA_aligned_sort_mkdp
  Keeping 4 of 40780 bins
  Correcting for GC bias...
  Correcting for RepeatMasker bias...
  WARNING: Most antitarget bins (100.00%, 4/4) have low or no coverage; is this amplicon/WGS?
  /usr/local/lib/python3.10/site-packages/cnvlib/cnary.py:492: FutureWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
    return pd.concat(resids) if resids else pd.Series([])
  Antitargets are nan x more variable than targets
  Wrote ./test_tumor_DNA_aligned_sort_mkdp.cnr with 281 regions
  Segmenting ./test_tumor_DNA_aligned_sort_mkdp.cnr ...
  Segmenting with method 'cbs', significance threshold 0.0001, in 16 processes
  Traceback (most recent call last):
    File "/usr/local/bin/cnvkit.py", line 10, in <module>
      sys.exit(main())
    File "/usr/local/lib/python3.10/site-packages/cnvlib/cnvkit.py", line 10, in main
      args.func(args)
    File "/usr/local/lib/python3.10/site-packages/cnvlib/commands.py", line 2075, in _cmd_segmetrics
      segarr = read_cna(args.segments)
    File "/usr/local/lib/python3.10/site-packages/cnvlib/cmdutil.py", line 12, in read_cna
      return tabio.read(infile, into=CNA, sample_id=sample_id, meta=meta)
    File "/usr/local/lib/python3.10/site-packages/skgenome/tabio/__init__.py", line 75, in read
      dframe = reader(infile, **kwargs)
    File "/usr/local/lib/python3.10/site-packages/skgenome/tabio/tab.py", line 17, in read_tab
      dframe = pd.read_csv(infile, sep="\t", dtype={"chromosome": "str"})
    File "/usr/local/lib/python3.10/site-packages/pandas/util/_decorators.py", line 211, in wrapper
      return func(*args, **kwargs)
    File "/usr/local/lib/python3.10/site-packages/pandas/util/_decorators.py", line 331, in wrapper
      return func(*args, **kwargs)
    File "/usr/local/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 950, in read_csv
      return _read(filepath_or_buffer, kwds)
    File "/usr/local/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 605, in _read
      parser = TextFileReader(filepath_or_buffer, **kwds)
    File "/usr/local/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1442, in __init__
      self._engine = self._make_engine(f, self.engine)
    File "/usr/local/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1735, in _make_engine
      self.handles = get_handle(
    File "/usr/local/lib/python3.10/site-packages/pandas/io/common.py", line 856, in get_handle
      handle = open(
  FileNotFoundError: [Errno 2] No such file or directory: 'test_tumor_DNA_aligned_sort_mkdp.cns'

Work dir:
  /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/cd/6aa76a9d0aa4297882852c614adf0a

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

I attached below the files from work dir: "cd/6aa76a9d0aa4297882852c614adf0a"

command.out.tar.gz command.log.tar.gz command.err.tar.gz command.run.tar.gz

Thanks

riederd commented 6 months ago

can you post the output of ls -la /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/cd/6aa76a9d0aa4297882852c614adf0a

What kind of data is it? WES, WGS or a targeted panel? TO bi sure, it is human, right?

rashidma commented 6 months ago

mamoon@DESKTOP-CFF6CD1:/mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/cd/6aa76a9d0aa4297882852c614adf0a$ ls -la

total 69000
drwxr-xr-x 1 mamoon mamoon     4096 Jan 10 16:16 .
drwxr-xr-x 1 mamoon mamoon     4096 Jan 10 16:01 ..
-rw-r--r-- 1 mamoon mamoon        0 Jan 10 16:11 .command.begin
-rw-r--r-- 1 mamoon mamoon     7531 Jan 10 16:16 .command.err
-rw-r--r-- 1 mamoon mamoon     7531 Jan 10 16:16 .command.log
-rw-r--r-- 1 mamoon mamoon        0 Jan 10 16:11 .command.out
-rw-r--r-- 1 mamoon mamoon    11950 Jan 10 16:11 .command.run
-rw-r--r-- 1 mamoon mamoon     1777 Jan 10 16:11 .command.sh
-rw-r--r-- 1 mamoon mamoon        0 Jan 10 16:11 .command.trace
-rw-r--r-- 1 mamoon mamoon        1 Jan 10 16:16 .exitcode
lrwxrwxrwx 1 mamoon mamoon       82 Jan 10 16:11 GRCh38.d1.vd1.fa -> /mnt/e/Mamoon/db_nextflow/references/hg38/gdc/GRCh38.d1.vd1/fasta/GRCh38.d1.vd1.fa
lrwxrwxrwx 1 mamoon mamoon       86 Jan 10 16:11 GRCh38.d1.vd1.fa.fai -> /mnt/e/Mamoon/db_nextflow/references/hg38/gdc/GRCh38.d1.vd1/fasta/GRCh38.d1.vd1.fa.fai
-rw-r--r-- 1 mamoon mamoon  1422362 Jan 10 16:14 S07604514_Covered.antitarget.bed
lrwxrwxrwx 1 mamoon mamoon       81 Jan 10 16:11 S07604514_Covered.bed -> /mnt/e/Mamoon/db_nextflow/ExomeCaptureKits/Agilent/hg38/S07604514_Covered_ann.bed
-rw-r--r-- 1 mamoon mamoon 13019076 Jan 10 16:14 S07604514_Covered.target.bed
lrwxrwxrwx 1 mamoon mamoon      112 Jan 10 16:11 access-5kb.GRCh38.bed -> /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/2d/9c7dacbfc621aa5b13944e0417b527/access-5kb.GRCh38.bed
lrwxrwxrwx 1 mamoon mamoon       96 Jan 10 16:11 gencode.v33.primary_assembly.annotation.gtf -> /mnt/e/Mamoon/db_nextflow/references/hg38/annotation/gencode.v33.primary_assembly.annotation.gtf
-rw-r--r-- 1 mamoon mamoon       14 Jan 10 16:11 matplotlibrc
-rw-r--r-- 1 mamoon mamoon 23229970 Jan 10 16:15 output_reference.cnn
-rw-r--r-- 1 mamoon mamoon  1693156 Jan 10 16:14 test_normal_DNA_aligned_sort_mkdp.antitargetcoverage.cnn
lrwxrwxrwx 1 mamoon mamoon      128 Jan 10 16:11 test_normal_DNA_aligned_sort_mkdp.bai -> /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/c3/ad562c927de630a227e9028575c363/test_normal_DNA_aligned_sort_mkdp.bai
lrwxrwxrwx 1 mamoon mamoon      128 Jan 10 16:11 test_normal_DNA_aligned_sort_mkdp.bam -> /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/c3/ad562c927de630a227e9028575c363/test_normal_DNA_aligned_sort_mkdp.bam
-rw-r--r-- 1 mamoon mamoon 14759907 Jan 10 16:14 test_normal_DNA_aligned_sort_mkdp.targetcoverage.cnn
-rw-r--r-- 1 mamoon mamoon  1699632 Jan 10 16:16 test_tumor_DNA_aligned_sort_mkdp.antitargetcoverage.cnn
lrwxrwxrwx 1 mamoon mamoon      127 Jan 10 16:11 test_tumor_DNA_aligned_sort_mkdp.bai -> /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/6d/1d954d7c42f1d93703c4538e708e6c/test_tumor_DNA_aligned_sort_mkdp.bai
lrwxrwxrwx 1 mamoon mamoon      127 Jan 10 16:11 test_tumor_DNA_aligned_sort_mkdp.bam -> /mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo/work/6d/1d954d7c42f1d93703c4538e708e6c/test_tumor_DNA_aligned_sort_mkdp.bam
-rw-r--r-- 1 mamoon mamoon    18729 Jan 10 16:16 test_tumor_DNA_aligned_sort_mkdp.cnr
-rw-r--r-- 1 mamoon mamoon 14762084 Jan 10 16:16 test_tumor_DNA_aligned_sort_mkdp.targetcoverage.cnn

The test data is as below: Test data is taken from: https://apps-01.i-med.ac.at/resources/nextneopi/nextNEOpi_testdata.tar.gz

mamoon@DESKTOP-CFF6CD1:/mnt/e/Mamoon/NRC21R-082-03/Analysis/nextNEOpi/demo$ ls -la testdata/
total 316376
drwxrwxrwx 1 root root     4096 Sep 21 18:37 .
drwxrwxrwx 1 root root     4096 Jan 10 15:35 ..
-rwxrwxrwx 1 root root 52692230 Sep 21 18:37 normal_DNA_R1.fastq.gz
-rwxrwxrwx 1 root root 52770181 Sep 21 18:36 normal_DNA_R2.fastq.gz
-rwxrwxrwx 1 root root      353 Sep 21 18:36 regions_4_test_dataset.txt
-rwxrwxrwx 1 root root 73599182 Sep 21 18:36 tumor_DNA_R1.fastq.gz
-rwxrwxrwx 1 root root 74170100 Sep 21 18:36 tumor_DNA_R2.fastq.gz
-rwxrwxrwx 1 root root 35029431 Sep 21 18:37 tumor_RNA_R1.fastq.gz
-rwxrwxrwx 1 root root 35695498 Sep 21 18:36 tumor_RNA_R2.fastq.gz

I realized the test data will not run successfully at "cnvit" due to limited read coverage. So, I am running the pipeline without "cnvkit". Let us see what happens. Thanks

riederd commented 2 months ago

No more feedback, assuming the issue as solved