Error on test code (however, using singularity)

gailrosen commented 1 year ago

Hi,

Getting all this to work in singularity is a nightmare since there are a lot of wrapper scripts around the docker. Singularity can pull a docker and work as long as everything is self-contained in the docker. Would love for that to be the case, where all these wrapper are inside the docker. I am having trouble running things -- so now I have tried the test.

I am running: singularity run -B /ifs/groups/rosenMRIGrp/NIST_datasets/fastvifi_depdata/kraken_datasets:/home/kraken2-db -B .:/test ../fastvifi_latest.sif python /test/run_kraken_vifi_pipeline.py --output-dir /test/fastvifi_output_files --input-file /test/test_reads_1.fq --input-file-2 /test/test_reads_2.fq --level sample-level-validation-intermediate --kraken-path /home/kraken2/kraken2 --kraken-db-path /home/kraken2-db --vifi-path /home/ViFi/scripts/run_vifi.py --virus hpv --human-chr-list /test/human_chr_list.txt --skip-bwa-filter --keep-intermediate-files

But the BAMs are not output -- do you know why? (see below even when i try to get rid of the internal docker flag). Could you make this a little more singularity friendly and have the pipelines contained within the container?

Here is the output:

Warning! output directory already exists. Rewriting previous outputs. Loading database information... done. 19 sequences (0.00 Mbp) processed in 0.006s (193.4 Kseq/m, 30.53 Mbp/m). 19 sequences classified (100.00%) 0 sequences unclassified (0.00%) Command being timed: "/home/kraken2/kraken2 --use-names --report /test/fastvifi_output_files/kraken_report_first_level_for_virus_hp v --db /home/kraken2-db/Kraken2StandardDB_k_25_hpv_hg --threads 1 --paired --f-threshold 0.4 --keep-unmapped-reads --unmapped-threshold 0.8 --classified-out /test/fastvifi_output_files/reads_passing_kraken_first_level_for_virus_hpv#.fq /test/fastvifi_output_files/input_file_1.f q /test/fastvifi_output_files/input_file_2.fq --output /test/fastvifi_output_files/kraken_output_classify_first_level_for_virus_hpv" User time (seconds): 0.01 System time (seconds): 2.43 Percent of CPU this job got: 98% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.50 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 4166152 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 30 Minor (reclaiming a frame) page faults: 1042986 Voluntary context switches: 495 Involuntary context switches: 85 Swaps: 0 File system inputs: 90402 File system outputs: 32 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 Loading database information... done. 10 sequences (0.00 Mbp) processed in 0.004s (143.3 Kseq/m, 42.99 Mbp/m). 10 sequences classified (100.00%) 0 sequences unclassified (0.00%) Command being timed: "/home/kraken2/kraken2 --use-names --report /test/fastvifi_output_files/kraken_report_final_level_for_virus_hpv --db /home/kraken2-db/Kraken2StandardDB_k_22_hpv --threads 1 --paired --f-threshold 0.5 --unmapped-threshold 0.9 --classified-out /test/fastvifi_output_files/reads_passing_kraken_filter_for_virus_hpv#.fq /test/fastvifi_output_files/reads_passing_kraken_first_level_for_virus_hpv_1.fq /test/fastvifi_output_files/reads_passing_kraken_first_level_for_virus_hpv_2.fq --output /test/fastvifi_output_files/kraken_output_classify_final_level_for_virus_hpv" User time (seconds): 0.01 System time (seconds): 0.01 Percent of CPU this job got: 70% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.03 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 9796 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 6 Minor (reclaiming a frame) page faults: 3175 Voluntary context switches: 25 Involuntary context switches: 3 Swaps: 0 File system inputs: 14 File system outputs: 32 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 sh: 1: docker: not found
Command being timed: "python /home/ViFi/scripts/run_vifi.py --docker -f /test/fastvifi_output_files/reads_passing_kraken_filter_for_virus_hpv_1.fq -r /test/fastvifi_output_files/reads_passing_kraken_filter_for_virus_hpv_2.fq -o /test/fastvifi_output_files --virus hpv -c 1 --prefix output_hpv" User time (seconds): 0.01 System time (seconds): 0.03 Percent of CPU this job got: 94% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.05 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 11156 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 15 Minor (reclaiming a frame) page faults: 1880 Voluntary context switches: 25 Involuntary context switches: 7 Swaps: 0 File system inputs: 274 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [E::hts_open_format] Failed to open file "/test/fastvifi_output_files/output_hpv.viral.bam" : No such file or directory Traceback (most recent call last): File "/test/run_kraken_vifi_pipeline.py", line 468, in run_pipeline(args) File "/test/run_kraken_vifi_pipeline.py", line 459, in run_pipeline bwa_filtered_fq_filename_2=bwa_filtered_fq_filename_2) File "/test/run_kraken_vifi_pipeline.py", line 326, in run_kraken_vifi vifi_input_fq_1=vifi_input_fq_1) File "/test/run_kraken_vifi_pipeline.py", line 358, in write_viral_reads_report viral_aligned_file = pysam.AlignmentFile(os.path.join(args.output_dir, "{}.viral.bam".format(prefix)), 'rb') File "pysam/libcalignmentfile.pyx", line 751, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 950, in pysam.libcalignmentfile.AlignmentFile._open IOError: [Errno 2] could not open alignment file /test/fastvifi_output_files/output_hpv.viral.bam: No such file or directory [gailr@node040 vifi_test]$ singularity run -B /ifs/groups/rosenMRIGrp/NIST_datasets/fastvifi_depdata/kraken_datasets:/home/kraken2-db -B .:/test ../fastvifi_latest.sif python /test/run_kraken_vifi_pipeline.py --output-dir /test/fastvifi_output_files --input-file /test/test_reads_1.fq --input-file-2 /test/test_reads_2.fq --level sample-level-validation-intermediate --kraken-path /home/kraken2/kraken2 --kraken-db-path /home/kraken2-db --vifi-path /home/ViFi/scripts/run_vifi.py --virus hpv --human-chr-list /test/human_chr_list.txt --skip-bwa-filter --keep-intermediate-files |more Loading database information... done. 19 sequences (0.00 Mbp) processed in 0.006s (191.5 Kseq/m, 30.24 Mbp/m). 19 sequences classified (100.00%) 0 sequences unclassified (0.00%) Command being timed: "/home/kraken2/kraken2 --use-names --report /test/fastvifi_output_files/kraken_report_first_level_for_virus_hpv --db /home/kraken2-db/Kraken2StandardDB_k_25_hpv_hg --threads 1 --paired --f-threshold 0.4 --keep-unmapped-reads --unmapped-threshold 0.8 --classified-out /test/fastvifi_output_files/reads_passing_kraken_first_level_for_virus_hpv#.fq /test/fastvifi_output_files/input_file_1.fq /test/fastvifi_output_files/input_file_2.fq --output /test/fastvifi_output_files/kraken_output_classify_first_level_for_virus_hpv" User time (seconds): 0.01 System time (seconds): 2.41 Percent of CPU this job got: 98% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.46 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 4166256 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 30 Minor (reclaiming a frame) page faults: 1042987 Voluntary context switches: 418 Involuntary context switches: 84 Swaps: 0 File system inputs: 60410 File system outputs: 32 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 Loading database information... done. 10 sequences (0.00 Mbp) processed in 0.005s (119.9 Kseq/m, 35.96 Mbp/m). 10 sequences classified (100.00%) 0 sequences unclassified (0.00%) Command being timed: "/home/kraken2/kraken2 --use-names --report /test/fastvifi_output_files/kraken_report_final_level_for_virus_hpv --db /home/kraken2-db/Kraken2StandardDB_k_22_hpv --threads 1 --paired --f-threshold 0.5 --unmapped-threshold 0.9 --classified-out /test/fastvifi_output_files/reads_passing_kraken_filter_for_virus_hpv#.fq /test/fastvifi_output_files/reads_passing_kraken_first_level_for_virus_hpv_1.fq /test/fastvifi_output_files/reads_passing_kraken_first_level_for_virus_hpv_2.fq --output /test/fastvifi_output_files/kraken_output_classify_final_level_for_virus_hpv" User time (seconds): 0.01 System time (seconds): 0.00 Percent of CPU this job got: 67% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.03 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 9796 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 4 Minor (reclaiming a frame) page faults: 3174 Voluntary context switches: 25 Involuntary context switches: 5 Swaps: 0 File system inputs: 14 File system outputs: 32 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 sh: 1: docker: not found
Command being timed: "python /home/ViFi/scripts/run_vifi.py --docker -f /test/fastvifi_output_files/reads_passing_kraken_filter_for_virus_hpv_1.fq -r /test/fastvifi_output_files/reads_passing_kraken_filter_for_virus_hpv_2.fq -o /test/fastvifi_output_files --virus hpv -c 1 --prefix output_hpv" User time (seconds): 0.01 System time (seconds): 0.03 Percent of CPU this job got: 94% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.05 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 11156 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 15 Minor (reclaiming a frame) page faults: 1880 Voluntary context switches: 25 Involuntary context switches: 5 Swaps: 0 File system inputs: 274 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [E::hts_open_format] Failed to open file "/test/fastvifi_output_files/output_hpv.viral.bam" : No such file or directory Traceback (most recent call last): File "/test/run_kraken_vifi_pipeline.py", line 468, in run_pipeline(args) File "/test/run_kraken_vifi_pipeline.py", line 459, in run_pipeline bwa_filtered_fq_filename_2=bwa_filtered_fq_filename_2) File "/test/run_kraken_vifi_pipeline.py", line 326, in run_kraken_vifi vifi_input_fq_1=vifi_input_fq_1) File "/test/run_kraken_vifi_pipeline.py", line 358, in write_viral_reads_report viral_aligned_file = pysam.AlignmentFile(os.path.join(args.output_dir, "{}.viral.bam".format(prefix)), 'rb') File "pysam/libcalignmentfile.pyx", line 751, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 950, in pysam.libcalignmentfile.AlignmentFile._open IOError: [Errno 2] could not open alignment file /test/fastvifi_output_files/output_hpv.viral.bam: No such file or directory Warning! output directory already exists. Rewriting previous outputs.

I tried getting rid of the docker flag in the run pipeline script and got:

[gailr@node040 vifi_test]$ singularity run -B /ifs/groups/rosenMRIGrp/NIST_datasets/fastvifi_depdata/kraken_datasets:/home/kraken2-db -B .: /test ../fastvifi_latest.sif python /test/run_kraken_vifi_pipeline.py --output-dir /test/fastvifi_output_files --input-file /test/test_read s_1.fq --input-file-2 /test/test_reads_2.fq --level sample-level-validation-intermediate --kraken-path /home/kraken2/kraken2 --kraken-db-pa th /home/kraken2-db --vifi-path /home/ViFi/scripts/run_vifi.py --virus hpv --human-chr-list /test/human_chr_list.txt --skip-bwa-filter --ke ep-intermediate-files Warning! output directory already exists. Rewriting previous outputs. Loading database information... done. 19 sequences (0.00 Mbp) processed in 0.007s (155.5 Kseq/m, 24.56 Mbp/m). 19 sequences classified (100.00%) 0 sequences unclassified (0.00%) Command being timed: "/home/kraken2/kraken2 --use-names --report /test/fastvifi_output_files/kraken_report_first_level_for_virus_hp v --db /home/kraken2-db/Kraken2StandardDB_k_25_hpv_hg --threads 1 --paired --f-threshold 0.4 --keep-unmapped-reads --unmapped-threshold 0.8 --classified-out /test/fastvifi_output_files/reads_passing_kraken_first_level_for_virus_hpv#.fq /test/fastvifi_output_files/input_file_1.f q /test/fastvifi_output_files/input_file_2.fq --output /test/fastvifi_output_files/kraken_output_classify_first_level_for_virus_hpv" User time (seconds): 0.01 System time (seconds): 2.35 Percent of CPU this job got: 87% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.70 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 4166328 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 30 Minor (reclaiming a frame) page faults: 1042985 Voluntary context switches: 175 Involuntary context switches: 104 Swaps: 0 File system inputs: 60074 File system outputs: 32 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 Loading database information... done. 10 sequences (0.00 Mbp) processed in 0.005s (117.1 Kseq/m, 35.12 Mbp/m). 10 sequences classified (100.00%) 0 sequences unclassified (0.00%) Command being timed: "/home/kraken2/kraken2 --use-names --report /test/fastvifi_output_files/kraken_report_final_level_for_virus_hpv --db /home/kraken2-db/Kraken2StandardDB_k_22_hpv --threads 1 --paired --f-threshold 0.5 --unmapped-threshold 0.9 --classified-out /test/fastvifi_output_files/reads_passing_kraken_filter_for_virus_hpv#.fq /test/fastvifi_output_files/reads_passing_kraken_first_level_for_virus_hpv_1.fq /test/fastvifi_output_files/reads_passing_kraken_first_level_for_virus_hpv_2.fq --output /test/fastvifi_output_files/kraken_output_classify_final_level_for_virus_hpv" User time (seconds): 0.01 System time (seconds): 0.01 Percent of CPU this job got: 70% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.04 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 9604 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 12 Minor (reclaiming a frame) page faults: 2155 Voluntary context switches: 24 Involuntary context switches: 4 Swaps: 0 File system inputs: 14 File system outputs: 32 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [E::bwa_idx_load_from_disk] fail to locate the index files Traceback (most recent call last): File "/home/ViFi/scripts/get_trans_new.py", line 106, in bamFile = pysam.Samfile(opts.dataName[0], 'rb') File "pysam/libcalignmentfile.pyx", line 751, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 1000, in pysam.libcalignmentfile.AlignmentFile._open ValueError: file has no sequences defined (mode='rb') - is it SAM/BAM format? Consider opening with check_sq=False ls: cannot access '/home/repo/data/hpv/hmms/*.hmmbuild': No such file or directory [E::hts_open_format] Failed to open file "/test/fastvifi_output_files/output_hpv.unknown.bam" : No such file or directory Traceback (most recent call last): File "/home/ViFi/scripts/run_hmms.py", line 112, in run_pipeline(options)
File "/home/ViFi/scripts/run_hmms.py", line 37, in run_pipeline prepare_unmapped_sequences(options) File "/home/ViFi/scripts/run_hmms.py", line 95, in prepare_unmapped_sequences bam = pysam.Samfile(options.bamfile, 'rb') File "pysam/libcalignmentfile.pyx", line 751, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 950, in pysam.libcalignmentfile.AlignmentFile._open IOError: [Errno 2] could not open alignment file /test/fastvifi_output_files/output_hpv.unknown.bam: No such file or directory [E::hts_open_format] Failed to open file "/test/fastvifi_output_files/output_hpv.trans.bam" : No such file or directory Traceback (most recent call last): File "/home/ViFi/scripts/merge_viral_reads.py", line 117, in transFile = pysam.Samfile(args.transName[0], 'rb') File "pysam/libcalignmentfile.pyx", line 751, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 950, in pysam.libcalignmentfile.AlignmentFile._open IOError: [Errno 2] could not open alignment file /test/fastvifi_output_files/output_hpv.trans.bam: No such file or directory [E::hts_open_format] Failed to open file /test/fastvifi_output_files/output_hpv.fixed.trans.bam samtools sort: can't open "/test/fastvifi_output_files/output_hpv.fixed.trans.bam": No such file or directory samtools index: "/test/fastvifi_output_files/output_hpv.fixed.trans.cs.bam" is in a format that cannot be usefully indexed [E::hts_open_format] Failed to open file /test/fastvifi_output_files/output_hpv.viral.bam samtools sort: can't open "/test/fastvifi_output_files/output_hpv.viral.bam": No such file or directory samtools index: "/test/fastvifi_output_files/output_hpv.viral.cs.bam" is in a format that cannot be usefully indexed WARNING:root:#TIME 0.042 Unable to find reference in $AA_DATA_REPO/reference.txt. Setting to working directory. WARNING:root:#TIME 0.044 Unable to find reference in $AA_DATA_REPO/REF/file_list.txt. Setting to empty. WARNING:root:#TIME 0.044 Unable to open fasta file: "/home/data_repo//". Reference sequences will be set to N. WARNING:root:#TIME 0.044 Unable to open chromosome lengths file: "/home/data_repo//" WARNING:root:#TIME 0.044 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.044 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.044 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.044 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.044 interval_list: Unable to open interval file "/home/data_repo//". Traceback (most recent call last): File "/home/ViFi/scripts/cluster_trans_new.py", line 38, in bamFile = pysam.Samfile(args.dataName[0], 'rb') File "pysam/libcalignmentfile.pyx", line 751, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 956, in pysam.libcalignmentfile.AlignmentFile._open ValueError: file does not contain alignment data Command being timed: "python /home/ViFi/scripts/run_vifi.py -f /test/fastvifi_output_files/reads_passing_kraken_filter_for_virus_hpv_1.fq -r /test/fastvifi_output_files/reads_passing_kraken_filter_for_virus_hpv_2.fq -o /test/fastvifi_output_files --virus hpv -c 1 --prefix output_hpv" User time (seconds): 0.14 System time (seconds): 0.17 Percent of CPU this job got: 67% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.47 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 23808 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 96 Minor (reclaiming a frame) page faults: 17524 Voluntary context switches: 171 Involuntary context switches: 90 Swaps: 0 File system inputs: 4512 File system outputs: 8 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [E::hts_open_format] Failed to open file "/test/fastvifi_output_files/output_hpv.viral.bam" : No such file or directory Traceback (most recent call last): File "/test/run_kraken_vifi_pipeline.py", line 469, in run_pipeline(args) File "/test/run_kraken_vifi_pipeline.py", line 460, in run_pipeline bwa_filtered_fq_filename_2=bwa_filtered_fq_filename_2) File "/test/run_kraken_vifi_pipeline.py", line 327, in run_kraken_vifi vifi_input_fq_1=vifi_input_fq_1) File "/test/run_kraken_vifi_pipeline.py", line 359, in write_viral_reads_report viral_aligned_file = pysam.AlignmentFile(os.path.join(args.output_dir, "{}.viral.bam".format(prefix)), 'rb') File "pysam/libcalignmentfile.pyx", line 751, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 950, in pysam.libcalignmentfile.AlignmentFile._open IOError: [Errno 2] could not open alignment file /test/fastvifi_output_files/output_hpv.viral.bam: No such file or directory

sara-javadzadeh commented 1 year ago

Hi Gail,

Thanks for reaching out. This is a good point. Let me look into this and get back to you.

sara-javadzadeh commented 10 months ago

Hi Gail,

I have added the option to run FastViFi with Singularity. Please refer to section "Running FastViFi with Singularity" of the manual, as you need to run run_kraken_vifi_container.py instead of run_kraken_vifi_pipeline.py.

I tested it with singularity version 3 or above installed. Please check the version of singularity by running singularity --version. Please let me know if you have any questions.

Sara

gailrosen commented 10 months ago

thanks! i'll try to check it out soon

From: Sara Javadzadeh @.***> Sent: Friday, August 11, 2023 4:57 PM To: sara-javadzadeh/FastViFi Cc: Rosen,Gail; Author Subject: Re: [sara-javadzadeh/FastViFi] Error on test code (however, using singularity) (Issue #10)

External.

Hi Gail,

I have added the option to run FastViFi with Singularity. Please refer to section "Running FastViFi with Singularity" of the manualhttps://github.com/sara-javadzadeh/FastViFi#running-fastvifi-with-singularity, as you need to run run_kraken_vifi_container.py instead of run_kraken_vifi_pipeline.py.

I tested it with singularity version 3 or above installed. Please check the version of singularity by running singularity --version. Please let me know if you have any questions.

Sara

— Reply to this email directly, view it on GitHubhttps://github.com/sara-javadzadeh/FastViFi/issues/10#issuecomment-1675395577, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABHZROOVINNEAZ6D3SRIB2TXU2MEVANCNFSM6AAAAAAY46H2XU. You are receiving this because you authored the thread.Message ID: @.***>

gailrosen commented 9 months ago

Hi,

It kind of stalled out last night when making the SIF. I came back in the morning and it was hung after like 10 hours.

However, I seem to be able to get into the singularity container.

But when I run the code, I get this:

python3 run_kraken_vifi_container.py --singularity --skip-bwa-filter --input-file /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_1.fq --input-file-2 /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq --output-dir test --virus hbv --kraken-db-path /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets --vifi-viral-ref-dir /ifs/groups/rosenMRIGrp/gailr/FastViFi/viral_data --human-chr-list /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/human_chr_list.txt --vifi-human-ref-dir /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo Changing the mode of the output directory to be writable by other users (i.e., the user in the docker container) singularity run --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test:/home/input/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq:/home/input/test_reads_2.fq --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets:/home/kraken2-db --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo:/home/data_repo/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/viral_data:/home/repo/data/ --bind test:/home/output/ --bind ./run_kraken_vifi_pipeline.py:/home/fastvifi/run_kraken_vifi_pipeline.py fastvifi_v1.1.sif python /home/fastvifi/run_kraken_vifi_pipeline.py --kraken-path /home/kraken2/kraken2 --vifi-path /home/ViFi/scripts/run_vifi.py --output /home/output --human-chr-list /home/data_repo/GRCh38/chrom_list.txt --kraken-db-path /home/kraken2-db --docker --virus hbv --input-file /home/input/test_reads_1.fq --input-file-2 /home/input/test_reads_2.fq --skip-bwa-filter Traceback (most recent call last): File "run_kraken_vifi_container.py", line 95, in call_fastvifi_pipeline(args) File "run_kraken_vifi_container.py", line 88, in call_fastvifi_pipeline shell_output = subprocess.check_output( File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 411, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 512, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command ' singularity run --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test:/home/input/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq:/home/input/test_reads_2.fq --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets:/home/kraken2-db --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo:/home/data_repo/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/viral_data:/home/repo/data/ --bind test:/home/output/ --bind ./run_kraken_vifi_pipeline.py:/home/fastvifi/run_kraken_vifi_pipeline.py fastvifi_v1.1.sif python /home/fastvifi/run_kraken_vifi_pipeline.py --kraken-path /home/kraken2/kraken2 --vifi-path /home/ViFi/scripts/run_vifi.py --output /home/output --human-chr-list /home/data_repo/GRCh38/chrom_list.txt --kraken-db-path /home/kraken2-db --docker --virus hbv --input-file /home/input/test_reads_1.fq --input-file-2 /home/input/test_reads_2.fq --skip-bwa-filter ' returned non-zero exit status 1.

From: Rosen,Gail @.***> Sent: Friday, August 11, 2023 4:59 PM To: sara-javadzadeh/FastViFi Subject: Re: [sara-javadzadeh/FastViFi] Error on test code (however, using singularity) (Issue #10)

thanks! i'll try to check it out soon

From: Sara Javadzadeh @.***> Sent: Friday, August 11, 2023 4:57 PM To: sara-javadzadeh/FastViFi Cc: Rosen,Gail; Author Subject: Re: [sara-javadzadeh/FastViFi] Error on test code (however, using singularity) (Issue #10)

External.

Hi Gail,

I have added the option to run FastViFi with Singularity. Please refer to section "Running FastViFi with Singularity" of the manualhttps://github.com/sara-javadzadeh/FastViFi#running-fastvifi-with-singularity, as you need to run run_kraken_vifi_container.py instead of run_kraken_vifi_pipeline.py.

I tested it with singularity version 3 or above installed. Please check the version of singularity by running singularity --version. Please let me know if you have any questions.

Sara

— Reply to this email directly, view it on GitHubhttps://github.com/sara-javadzadeh/FastViFi/issues/10#issuecomment-1675395577, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABHZROOVINNEAZ6D3SRIB2TXU2MEVANCNFSM6AAAAAAY46H2XU. You are receiving this because you authored the thread.Message ID: @.***>

gailrosen commented 9 months ago

I have figured out that the Kraken database is not available in Kraken_datasets:

Warning! output directory already exists. Rewriting previous outputs. Error: Kraken database /home/kraken2-db/Kraken2StandardDB_k_25_hbv_hg not created. Create the database with corresponding k-mer length and indices.

Do I need to make this?

From: Rosen,Gail @.***> Sent: Tuesday, October 3, 2023 10:54 AM To: sara-javadzadeh/FastViFi Subject: Re: [sara-javadzadeh/FastViFi] Error on test code (however, using singularity) (Issue #10)

Hi,

It kind of stalled out last night when making the SIF. I came back in the morning and it was hung after like 10 hours.

However, I seem to be able to get into the singularity container.

But when I run the code, I get this:

python3 run_kraken_vifi_container.py --singularity --skip-bwa-filter --input-file /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_1.fq --input-file-2 /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq --output-dir test --virus hbv --kraken-db-path /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets --vifi-viral-ref-dir /ifs/groups/rosenMRIGrp/gailr/FastViFi/viral_data --human-chr-list /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/human_chr_list.txt --vifi-human-ref-dir /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo Changing the mode of the output directory to be writable by other users (i.e., the user in the docker container) singularity run --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test:/home/input/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq:/home/input/test_reads_2.fq --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets:/home/kraken2-db --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo:/home/data_repo/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/viral_data:/home/repo/data/ --bind test:/home/output/ --bind ./run_kraken_vifi_pipeline.py:/home/fastvifi/run_kraken_vifi_pipeline.py fastvifi_v1.1.sif python /home/fastvifi/run_kraken_vifi_pipeline.py --kraken-path /home/kraken2/kraken2 --vifi-path /home/ViFi/scripts/run_vifi.py --output /home/output --human-chr-list /home/data_repo/GRCh38/chrom_list.txt --kraken-db-path /home/kraken2-db --docker --virus hbv --input-file /home/input/test_reads_1.fq --input-file-2 /home/input/test_reads_2.fq --skip-bwa-filter Traceback (most recent call last): File "run_kraken_vifi_container.py", line 95, in call_fastvifi_pipeline(args) File "run_kraken_vifi_container.py", line 88, in call_fastvifi_pipeline shell_output = subprocess.check_output( File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 411, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 512, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command ' singularity run --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test:/home/input/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq:/home/input/test_reads_2.fq --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets:/home/kraken2-db --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo:/home/data_repo/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/viral_data:/home/repo/data/ --bind test:/home/output/ --bind ./run_kraken_vifi_pipeline.py:/home/fastvifi/run_kraken_vifi_pipeline.py fastvifi_v1.1.sif python /home/fastvifi/run_kraken_vifi_pipeline.py --kraken-path /home/kraken2/kraken2 --vifi-path /home/ViFi/scripts/run_vifi.py --output /home/output --human-chr-list /home/data_repo/GRCh38/chrom_list.txt --kraken-db-path /home/kraken2-db --docker --virus hbv --input-file /home/input/test_reads_1.fq --input-file-2 /home/input/test_reads_2.fq --skip-bwa-filter ' returned non-zero exit status 1.

From: Rosen,Gail @.***> Sent: Friday, August 11, 2023 4:59 PM To: sara-javadzadeh/FastViFi Subject: Re: [sara-javadzadeh/FastViFi] Error on test code (however, using singularity) (Issue #10)

thanks! i'll try to check it out soon

From: Sara Javadzadeh @.***> Sent: Friday, August 11, 2023 4:57 PM To: sara-javadzadeh/FastViFi Cc: Rosen,Gail; Author Subject: Re: [sara-javadzadeh/FastViFi] Error on test code (however, using singularity) (Issue #10)

External.

Hi Gail,

I have added the option to run FastViFi with Singularity. Please refer to section "Running FastViFi with Singularity" of the manualhttps://github.com/sara-javadzadeh/FastViFi#running-fastvifi-with-singularity, as you need to run run_kraken_vifi_container.py instead of run_kraken_vifi_pipeline.py.

I tested it with singularity version 3 or above installed. Please check the version of singularity by running singularity --version. Please let me know if you have any questions.

Sara

— Reply to this email directly, view it on GitHubhttps://github.com/sara-javadzadeh/FastViFi/issues/10#issuecomment-1675395577, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABHZROOVINNEAZ6D3SRIB2TXU2MEVANCNFSM6AAAAAAY46H2XU. You are receiving this because you authored the thread.Message ID: @.***>

gailrosen commented 9 months ago

Even if I just run against HPV,

I get the following errors:

    Involuntary context switches: 4
    Swaps: 0
    File system inputs: 10072
    File system outputs: 24
    Socket messages sent: 0
    Socket messages received: 0
    Signals delivered: 0
    Page size (bytes): 4096
    Exit status: 0

[E::hts_open_format] Failed to open file "/home/output/output_hpv.viral.bam" : No such file or directory Traceback (most recent call last): File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 491, in run_pipeline(args) File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 482, in run_pipeline bwa_filtered_fq_filename_2=bwa_filtered_fq_filename_2) File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 348, in run_kraken_vifi vifi_input_fq_1=vifi_input_fq_1) File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 380, in write_viral_reads_report viral_aligned_file = pysam.AlignmentFile(os.path.join(args.output_dir, "{}.viral.bam".format(prefix)), 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/output/output_hpv.viral.bam: No such file or directory Traceback (most recent call last): File "run_kraken_vifi_container.py", line 95, in call_fastvifi_pipeline(args) File "run_kraken_vifi_container.py", line 88, in call_fastvifi_pipeline shell_output = subprocess.check_output( File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 411, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 512, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command ' singularity run --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test:/home/input/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq:/home/input/test_reads_2.fq --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets:/home/kraken2-db --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo:/home/data_repo/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/viral_data:/home/repo/data/ --bind test:/home/output/ --bind ./run_kraken_vifi_pipeline.py:/home/fastvifi/run_kraken_vifi_pipeline.py fastvifi_v1.1.sif python /home/fastvifi/run_kraken_vifi_pipeline.py --kraken-path /home/kraken2/kraken2 --vifi-path /home/ViFi/scripts/run_vifi.py --output /home/output --human-chr-list /home/data_repo/GRCh38/chrom_list.txt --kraken-db-path /home/kraken2-db --docker --virus hpv --input-file /home/input/test_reads_1.fq --input-file-2 /home/input/test_reads_2.fq --skip-bwa-filter ' returned non-zero exit status 1.

From: Rosen,Gail @.***> Sent: Tuesday, October 3, 2023 1:24 PM To: sara-javadzadeh/FastViFi Subject: Re: [sara-javadzadeh/FastViFi] Error on test code (however, using singularity) (Issue #10)

I have figured out that the Kraken database is not available in Kraken_datasets:

Warning! output directory already exists. Rewriting previous outputs. Error: Kraken database /home/kraken2-db/Kraken2StandardDB_k_25_hbv_hg not created. Create the database with corresponding k-mer length and indices.

Do I need to make this?

From: Rosen,Gail @.***> Sent: Tuesday, October 3, 2023 10:54 AM To: sara-javadzadeh/FastViFi Subject: Re: [sara-javadzadeh/FastViFi] Error on test code (however, using singularity) (Issue #10)

Hi,

It kind of stalled out last night when making the SIF. I came back in the morning and it was hung after like 10 hours.

However, I seem to be able to get into the singularity container.

But when I run the code, I get this:

python3 run_kraken_vifi_container.py --singularity --skip-bwa-filter --input-file /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_1.fq --input-file-2 /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq --output-dir test --virus hbv --kraken-db-path /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets --vifi-viral-ref-dir /ifs/groups/rosenMRIGrp/gailr/FastViFi/viral_data --human-chr-list /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/human_chr_list.txt --vifi-human-ref-dir /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo Changing the mode of the output directory to be writable by other users (i.e., the user in the docker container) singularity run --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test:/home/input/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq:/home/input/test_reads_2.fq --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets:/home/kraken2-db --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo:/home/data_repo/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/viral_data:/home/repo/data/ --bind test:/home/output/ --bind ./run_kraken_vifi_pipeline.py:/home/fastvifi/run_kraken_vifi_pipeline.py fastvifi_v1.1.sif python /home/fastvifi/run_kraken_vifi_pipeline.py --kraken-path /home/kraken2/kraken2 --vifi-path /home/ViFi/scripts/run_vifi.py --output /home/output --human-chr-list /home/data_repo/GRCh38/chrom_list.txt --kraken-db-path /home/kraken2-db --docker --virus hbv --input-file /home/input/test_reads_1.fq --input-file-2 /home/input/test_reads_2.fq --skip-bwa-filter Traceback (most recent call last): File "run_kraken_vifi_container.py", line 95, in call_fastvifi_pipeline(args) File "run_kraken_vifi_container.py", line 88, in call_fastvifi_pipeline shell_output = subprocess.check_output( File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 411, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 512, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command ' singularity run --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test:/home/input/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq:/home/input/test_reads_2.fq --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets:/home/kraken2-db --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo:/home/data_repo/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/viral_data:/home/repo/data/ --bind test:/home/output/ --bind ./run_kraken_vifi_pipeline.py:/home/fastvifi/run_kraken_vifi_pipeline.py fastvifi_v1.1.sif python /home/fastvifi/run_kraken_vifi_pipeline.py --kraken-path /home/kraken2/kraken2 --vifi-path /home/ViFi/scripts/run_vifi.py --output /home/output --human-chr-list /home/data_repo/GRCh38/chrom_list.txt --kraken-db-path /home/kraken2-db --docker --virus hbv --input-file /home/input/test_reads_1.fq --input-file-2 /home/input/test_reads_2.fq --skip-bwa-filter ' returned non-zero exit status 1.

From: Rosen,Gail @.***> Sent: Friday, August 11, 2023 4:59 PM To: sara-javadzadeh/FastViFi Subject: Re: [sara-javadzadeh/FastViFi] Error on test code (however, using singularity) (Issue #10)

thanks! i'll try to check it out soon

From: Sara Javadzadeh @.***> Sent: Friday, August 11, 2023 4:57 PM To: sara-javadzadeh/FastViFi Cc: Rosen,Gail; Author Subject: Re: [sara-javadzadeh/FastViFi] Error on test code (however, using singularity) (Issue #10)

External.

Hi Gail,

I have added the option to run FastViFi with Singularity. Please refer to section "Running FastViFi with Singularity" of the manualhttps://github.com/sara-javadzadeh/FastViFi#running-fastvifi-with-singularity, as you need to run run_kraken_vifi_container.py instead of run_kraken_vifi_pipeline.py.

I tested it with singularity version 3 or above installed. Please check the version of singularity by running singularity --version. Please let me know if you have any questions.

Sara

— Reply to this email directly, view it on GitHubhttps://github.com/sara-javadzadeh/FastViFi/issues/10#issuecomment-1675395577, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABHZROOVINNEAZ6D3SRIB2TXU2MEVANCNFSM6AAAAAAY46H2XU. You are receiving this because you authored the thread.Message ID: @.***>

gailrosen commented 9 months ago

just for reference, it is trying to open a file that was never made. Here are the files that were made: (pysam) [gailr@node042 FastViFi]$ ls -l test total 416 -rw-r--r-- 1 gailr rosenMRIGrp 4360 Oct 3 13:38 hmms.txt -rw-r--r-- 1 gailr rosenMRIGrp 138 Oct 2 19:23 human_chr_list.txt -rw-r--r-- 1 gailr rosenMRIGrp 6129 Oct 3 13:38 input_file_1.fq -rw-r--r-- 1 gailr rosenMRIGrp 6129 Oct 3 13:38 input_file_2.fq -rw-r--r-- 1 gailr rosenMRIGrp 1303 Oct 3 13:38 log_fastvifi.txt -rw-r--r-- 1 gailr rosenMRIGrp 321 Oct 3 13:38 log_fastvifi_verbose.txt -rw-r--r-- 1 gailr rosenMRIGrp 62 Oct 3 13:38 output_hpv.bam -rw-r--r-- 1 gailr rosenMRIGrp 0 Oct 3 13:38 output_hpv.fixed.trans.cs.bam -rw-r--r-- 1 gailr rosenMRIGrp 0 Oct 3 13:38 output_hpv.viral.cs.bam -rw-r--r-- 1 gailr rosenMRIGrp 3648 Oct 3 13:38 reads_passing_kraken_filter_for_virus_hpv_1.fq -rw-r--r-- 1 gailr rosenMRIGrp 3648 Oct 3 13:38 reads_passing_kraken_filter_for_virus_hpv_2.fq -rw-r--r-- 1 gailr rosenMRIGrp 3438 Oct 3 13:38 reads_passing_kraken_first_level_for_virus_hpv_1.fq -rw-r--r-- 1 gailr rosenMRIGrp 3438 Oct 3 13:38 reads_passing_kraken_first_level_for_virus_hpv_2.fq -rw-r--r-- 1 gailr rosenMRIGrp 1184 Oct 2 19:23 test_fastvifi.sh -rw-r--r-- 1 gailr rosenMRIGrp 6129 Oct 2 19:23 test_reads_1.fq -rw-r--r-- 1 gailr rosenMRIGrp 6129 Oct 2 19:23 test_reads_2.fq drwxrwsrwx 4 gailr rosenMRIGrp 44 Oct 3 13:25 tmp

gailrosen commented 9 months ago

for your reference, here is the full output

(pysam) [gailr@node042 FastViFi]$ python3 run_kraken_vifi_container.py --singularity --skip-bwa -filter --input-file /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_1.fq --input-file-2 /if s/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq --output-dir test --virus hpv --kr aken-db-path /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets --vifi-viral-ref-dir /ifs/group s/rosenMRIGrp/gailr/FastViFi/viral_data --human-chr-list /ifs/groups/rosenMRIGrp/gailr/FastViFi/tes t/human_chr_list.txt --vifi-human-ref-dir /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo > vifi_e rrors
Loading database information... done.
19 sequences (0.00 Mbp) processed in 0.005s (248.7 Kseq/m, 39.27 Mbp/m). 19 sequences classified (100.00%) 0 sequences unclassified (0.00%)
Command being timed: "/home/kraken2/kraken2 --use-names --report /dev/null --db /home/kraken2-d b/Kraken2StandardDB_k_25_hpv_hg --threads 1 --paired --f-threshold 0.4 --keep-unmapped-reads --unmapped -threshold 0.8 --classified-out /home/output/reads_passing_kraken_first_level_for_virus_hpv#.fq /home/o utput/input_file_1.fq /home/output/input_file_2.fq --output /dev/null" User time (seconds): 0.01
System time (seconds): 1.29 Percent of CPU this job got: 98% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.32 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 4167580 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 19 Minor (reclaiming a frame) page faults: 4558 Voluntary context switches: 145 Involuntary context switches: 60 Swaps: 0 File system inputs: 4094 File system outputs: 16 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 Loading database information... done. 10 sequences (0.00 Mbp) processed in 0.004s (144.8 Kseq/m, 43.43 Mbp/m). 10 sequences classified (100.00%) 0 sequences unclassified (0.00%)
Command being timed: "/home/kraken2/kraken2 --use-names --report /dev/null --db /home/kraken2-db/Kraken2StandardDB_k_22_hpv --threads 1 --paired --f-threshold 0.5 --unmapped-threshold 0.9 --classified-out /home/output/reads_passing_kraken_filter_for_virus_hpv#.fq /home/output/reads_passing_kraken_first_level_for_virus_hpv_1.fq /home/output/reads_passing_kraken_first_level_for_virus_hpv_2.fq --output /dev/null" User time (seconds): 0.01
System time (seconds): 0.00 Percent of CPU this job got: 66% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.02 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 9556 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 2085 Voluntary context switches: 18 Involuntary context switches: 0 Swaps: 0 File system inputs: 0 File system outputs: 16 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [E::bwa_idx_load_from_disk] fail to locate the index files Traceback (most recent call last):
File "/home/ViFi/scripts/get_trans_new.py", line 104, in bamFile = pysam.Samfile(opts.dataName[0], 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 996, in pysam.libcalignmentfile.AlignmentFile._open ValueError: file has no sequences defined (mode='rb') - is it SAM/BAM format? Consider opening with check_sq=False [E::hts_open_format] Failed to open file "/home/output/output_hpv.unknown.bam" : No such file or directory Traceback (most recent call last):
File "/home/ViFi/scripts/run_hmms.py", line 115, in run_pipeline(options) File "/home/ViFi/scripts/run_hmms.py", line 37, in run_pipeline prepare_unmapped_sequences(options) File "/home/ViFi/scripts/run_hmms.py", line 96, in prepare_unmapped_sequences bam = pysam.Samfile(options.bamfile, 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/output/output_hpv.unknown.bam: No such file or directory [E::hts_open_format] Failed to open file "/home/output/output_hpv.trans.bam" : No such file or directory Traceback (most recent call last):
File "/home/ViFi/scripts/merge_viral_reads.py", line 117, in transFile = pysam.Samfile(args.transName[0], 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/output/output_hpv.trans.bam: No such file or directory [E::hts_open_format] Failed to open file /home/output/output_hpv.fixed.trans.bam samtools sort: can't open "/home/output/output_hpv.fixed.trans.bam": No such file or directory samtools index: "/home/output/output_hpv.fixed.trans.cs.bam" is in a format that cannot be usefully indexed [E::hts_open_format] Failed to open file /home/output/output_hpv.viral.bam samtools sort: can't open "/home/output/output_hpv.viral.bam": No such file or directory samtools index: "/home/output/output_hpv.viral.cs.bam" is in a format that cannot be usefully indexed WARNING:root:#TIME 0.061 Unable to find reference in $AA_DATA_REPO/reference.txt. Setting to working directory. WARNING:root:#TIME 0.061 Unable to find reference in $AA_DATA_REPO/REF/file_list.txt. Setting to empty. [E::fai_build3_core] Failed to open the file /home/data_repo// WARNING:root:#TIME 0.061 Unable to open fasta file: "/home/data_repo//". Reference sequences will be set to N. WARNING:root:#TIME 0.061 Unable to open chromosome lengths file: "/home/data_repo//" WARNING:root:#TIME 0.061 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.061 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.061 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.061 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.061 interval_list: Unable to open interval file "/home/data_repo//". Traceback (most recent call last):
File "/home/ViFi/scripts/cluster_trans_new.py", line 37, in bamFile = pysam.Samfile(args.dataName[0], 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 952, in pysam.libcalignmentfile.AlignmentFile._open ValueError: file does not contain alignment data Command being timed: "python /home/ViFi/scripts/run_vifi.py -f /home/output/reads_passing_kraken_filter_for_virus_hpv_1.fq -r /home/output/reads_passing_kraken_filter_for_virus_hpv_2.fq -o /home/output --virus hpv -c 1 --prefix output_hpv" User time (seconds): 0.27
System time (seconds): 0.12 Percent of CPU this job got: 93% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.42 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 21024 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 48 Minor (reclaiming a frame) page faults: 17487 Voluntary context switches: 373 Involuntary context switches: 35 Swaps: 0 File system inputs: 10072
File system outputs: 24 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [E::hts_open_format] Failed to open file "/home/output/output_hpv.viral.bam" : No such file or directory Traceback (most recent call last):
File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 491, in run_pipeline(args) File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 482, in run_pipeline bwa_filtered_fq_filename_2=bwa_filtered_fq_filename_2) File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 348, in run_kraken_vifi vifi_input_fq_1=vifi_input_fq_1) File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 380, in write_viral_reads_report viral_aligned_file = pysam.AlignmentFile(os.path.join(args.output_dir, "{}.viral.bam".format(prefix)), 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/output/output_hpv.viral.bam: No such file or directory Traceback (most recent call last):
File "run_kraken_vifi_container.py", line 95, in call_fastvifi_pipeline(args) File "run_kraken_vifi_container.py", line 88, in call_fastvifi_pipeline shell_output = subprocess.check_output( File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 411, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 512, in run raise CalledProcessError(retcode, process.args, raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command ' singularity run --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test:/home/input/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq:/home/input/test_reads_2.fq --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets:/home/kraken2-db --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo:/home/data_repo/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/viral_data:/home/repo/data/ --bind test:/home/output/ --bind ./run_kraken_vifi_pipeline.py:/home/fastvifi/run_kraken_vifi_pipeline.py fastvifi_v1.1.sif python /home/fastvifi/run_kraken_vifi_pipeline.py --kraken-path /home/kraken2/kraken2 --vifi-path /home/ViFi/scripts/run_vifi.py --output /home/output --human-chr-list /home/data_repo/GRCh38/chrom_list.txt --kraken-db-path /home/kraken2-db --docker --virus hpv --input-file /home/input/test_reads_1.fq --input-file-2 /home/input/test_reads_2.fq --skip-bwa-filter ' returned non-zero exit status 1. (pysam) [gailr@node042 FastViFi]$

sara-javadzadeh commented 8 months ago

Hi Gail,

Thanks for providing all the output information. And sorry for the delay.

Did you confirm that the SIF file was created? It doesn't take 10 hours for me to run, so that part is a bit unexpected.

Assuming the SIF file was correctly created and looking through the output, it looks like kraken filters are run without error with 10 reads passing the sequences. You can find the reads in reads_passing_kraken_filter_for_virus_hpv_1.fq and reads_passing_kraken_filter_for_virus_hpv_2.fq if you are interested.

The error comes from the ViFi step. It looks like the reference data is not accessed correctly. Could you please double check that the reference files are present in the data_repo file?

The AA_DATA_REPO environment variable is set in the Docker file as the default variable for hg reference data. But passing a directory to the --vifi-human-ref-dir flag should replace the default variable. Plus, binding the directories through singularity, it should be accessible through the environment.

Best, Sara

On Tue, Oct 3, 2023 at 11:02 AM Gail Rosen @.***> wrote:

for your reference, here is the full output

(pysam) @. FastViFi]$ python3 run_kraken_vifi_container.py --singularity --skip-bwa -filter --input-file /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_1.fq --input-file-2 /if s/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq --output-dir test --virus hpv --kr aken-db-path /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets --vifi-viral-ref-dir /ifs/group s/rosenMRIGrp/gailr/FastViFi/viral_data --human-chr-list /ifs/groups/rosenMRIGrp/gailr/FastViFi/tes t/human_chr_list.txt --vifi-human-ref-dir /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo > vifi_e rrors Loading database information... done. 19 sequences (0.00 Mbp) processed in 0.005s (248.7 Kseq/m, 39.27 Mbp/m). 19 sequences classified (100.00%) 0 sequences unclassified (0.00%) Command being timed: "/home/kraken2/kraken2 --use-names --report /dev/null --db /home/kraken2-d b/Kraken2StandardDB_k_25_hpv_hg --threads 1 --paired --f-threshold 0.4 --keep-unmapped-reads --unmapped -threshold 0.8 --classified-out /home/output/reads_passing_kraken_first_level_for_virus_hpv#.fq /home/o utput/input_file_1.fq /home/output/input_file_2.fq --output /dev/null" User time (seconds): 0.01 System time (seconds): 1.29 Percent of CPU this job got: 98% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.32 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 4167580 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 19 Minor (reclaiming a frame) page faults: 4558 Voluntary context switches: 145 Involuntary context switches: 60 Swaps: 0 File system inputs: 4094 File system outputs: 16 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 Loading database information... done. 10 sequences (0.00 Mbp) processed in 0.004s (144.8 Kseq/m, 43.43 Mbp/m). 10 sequences classified (100.00%) 0 sequences unclassified (0.00%) Command being timed: "/home/kraken2/kraken2 --use-names --report /dev/null --db /home/kraken2-db/Kraken2StandardDB_k_22_hpv --threads 1 --paired --f-threshold 0.5 --unmapped-threshold 0.9 --classified-out /home/output/reads_passing_kraken_filter_for_virus_hpv#.fq /home/output/reads_passing_kraken_first_level_for_virus_hpv_1.fq /home/output/reads_passing_kraken_first_level_for_virus_hpv_2.fq --output /dev/null" User time (seconds): 0.01 System time (seconds): 0.00 Percent of CPU this job got: 66% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.02 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 9556 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 2085 Voluntary context switches: 18 Involuntary context switches: 0 Swaps: 0 File system inputs: 0 File system outputs: 16 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [E::bwa_idx_load_from_disk] fail to locate the index files Traceback (most recent call last): File "/home/ViFi/scripts/get_trans_new.py", line 104, in bamFile = pysam.Samfile(opts.dataName[0], 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 996, in pysam.libcalignmentfile.AlignmentFile._open ValueError: file has no sequences defined (mode='rb') - is it SAM/BAM format? Consider opening with check_sq=False [E::hts_open_format] Failed to open file "/home/output/output_hpv.unknown.bam" : No such file or directory Traceback (most recent call last): File "/home/ViFi/scripts/run_hmms.py", line 115, in run_pipeline(options) File "/home/ViFi/scripts/run_hmms.py", line 37, in run_pipeline prepare_unmapped_sequences(options) File "/home/ViFi/scripts/run_hmms.py", line 96, in prepare_unmapped_sequences bam = pysam.Samfile(options.bamfile, 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/output/output_hpv.unknown.bam: No such file or directory [E::hts_open_format] Failed to open file "/home/output/output_hpv.trans.bam" : No such file or directory Traceback (most recent call last): File "/home/ViFi/scripts/merge_viral_reads.py", line 117, in transFile = pysam.Samfile(args.transName[0], 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/output/output_hpv.trans.bam: No such file or directory [E::hts_open_format] Failed to open file /home/output/output_hpv.fixed.trans.bam samtools sort: can't open "/home/output/output_hpv.fixed.trans.bam": No such file or directory samtools index: "/home/output/output_hpv.fixed.trans.cs.bam" is in a format that cannot be usefully indexed [E::hts_open_format] Failed to open file /home/output/output_hpv.viral.bam samtools sort: can't open "/home/output/output_hpv.viral.bam": No such file or directory samtools index: "/home/output/output_hpv.viral.cs.bam" is in a format that cannot be usefully indexed WARNING:root:#TIME 0.061 Unable to find reference in $AA_DATA_REPO/reference.txt. Setting to working directory. WARNING:root:#TIME 0.061 Unable to find reference in $AA_DATA_REPO/REF/file_list.txt. Setting to empty. [E::fai_build3_core] Failed to open the file /home/data_repo// WARNING:root:#TIME 0.061 Unable to open fasta file: "/home/data_repo//". Reference sequences will be set to N. WARNING:root:#TIME 0.061 Unable to open chromosome lengths file: "/home/data_repo//" WARNING:root:#TIME 0.061 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.061 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.061 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.061 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.061 interval_list: Unable to open interval file "/home/data_repo//". Traceback (most recent call last): File "/home/ViFi/scripts/cluster_trans_new.py", line 37, in bamFile = pysam.Samfile(args.dataName[0], 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 952, in pysam.libcalignmentfile.AlignmentFile._open ValueError: file does not contain alignment data Command being timed: "python /home/ViFi/scripts/run_vifi.py -f /home/output/reads_passing_kraken_filter_for_virus_hpv_1.fq -r /home/output/reads_passing_kraken_filter_for_virus_hpv_2.fq -o /home/output --virus hpv -c 1 --prefix output_hpv" User time (seconds): 0.27 System time (seconds): 0.12 Percent of CPU this job got: 93% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.42 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 21024 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 48 Minor (reclaiming a frame) page faults: 17487 Voluntary context switches: 373 Involuntary context switches: 35 Swaps: 0 File system inputs: 10072 File system outputs: 24 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [E::hts_open_format] Failed to open file "/home/output/output_hpv.viral.bam" : No such file or directory Traceback (most recent call last): File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 491, in run_pipeline(args) File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 482, in run_pipeline bwa_filtered_fq_filename_2=bwa_filtered_fq_filename_2) File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 348, in run_kraken_vifi vifi_input_fq_1=vifi_input_fq_1) File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 380, in write_viral_reads_report viral_aligned_file = pysam.AlignmentFile(os.path.join(args.output_dir, "{}.viral.bam".format(prefix)), 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/output/output_hpv.viral.bam: No such file or directory Traceback (most recent call last): File "run_kraken_vifi_container.py", line 95, in call_fastvifi_pipeline(args) File "run_kraken_vifi_container.py", line 88, in call_fastvifi_pipeline shell_output = subprocess.check_output( File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 411, in check_output return run(popenargs, stdout=PIPE, timeout=timeout, check=True, File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 512, in run raise CalledProcessError(retcode, process.args, raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command ' singularity run --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test:/home/input/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq:/home/input/test_reads_2.fq --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets:/home/kraken2-db --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo:/home/data_repo/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/viral_data:/home/repo/data/ --bind test:/home/output/ --bind ./run_kraken_vifi_pipeline.py:/home/fastvifi/run_kraken_vifi_pipeline.py fastvifi_v1.1.sif python /home/fastvifi/run_kraken_vifi_pipeline.py --kraken-path /home/kraken2/kraken2 --vifi-path /home/ViFi/scripts/run_vifi.py --output /home/output --human-chr-list /home/data_repo/GRCh38/chrom_list.txt --kraken-db-path /home/kraken2-db --docker --virus hpv --input-file /home/input/test_reads_1.fq --input-file-2 /home/input/test_reads_2.fq --skip-bwa-filter ' returned non-zero exit status 1. (pysam) **@.*** FastViFi]$

— Reply to this email directly, view it on GitHub https://urldefense.com/v3/__https://github.com/sara-javadzadeh/FastViFi/issues/10*issuecomment-1745467395__;Iw!!Mih3wA!E2yNhpjfooMD1RN-W6BnKaLHHkoil6soEABfBTkWaaJZjK8Gd957a5RnA6fUwe2sZw_sag1rHN9ecz_rOxcNl1tPteI4RErC$, or unsubscribe https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AOGKYDSSZOMTUL2ZK6Y6HLTX5RHKXAVCNFSM6AAAAAAY46H2XWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBVGQ3DOMZZGU__;!!Mih3wA!E2yNhpjfooMD1RN-W6BnKaLHHkoil6soEABfBTkWaaJZjK8Gd957a5RnA6fUwe2sZw_sag1rHN9ecz_rOxcNl1tPtQxd362y$ . You are receiving this because you commented.Message ID: @.***>

gailrosen commented 8 months ago

(pysam) @. FastViFi]$ ls -l data_repo/ total 120 -rwxrwxrwx 1 gailr rosenMRIGrp 0 Apr 21 2020 coverage.stats drwxrwsrwx 5 gailr rosenMRIGrp 592 Feb 3 2021 GRCh37 drwxrwsrwx 4 gailr rosenMRIGrp 653 May 17 2020 GRCh38 drwxrwsrwx 5 gailr rosenMRIGrp 548 May 17 2020 hg19 (pysam) @. FastViFi]$ ls -l data_repo/hg19/ total 5321376 drwxrwsrwx 2 gailr rosenMRIGrp 41 Feb 3 2021 annotations drwxrwsrwx 3 gailr rosenMRIGrp 27 Mar 26 2017 cancer -rwxrwxrwx 1 gailr rosenMRIGrp 24896 Aug 5 2020 conserved.bed -rwxrwxrwx 1 gailr rosenMRIGrp 317 Dec 3 2019 dummy_ploidy.vcf -rwxrwxrwx 1 gailr rosenMRIGrp 682 Mar 31 2018 file_list.txt -rwxrwxrwx 1 gailr rosenMRIGrp 1643 Mar 26 2017 hg19_centromere.bed -rwxrwxrwx 1 gailr rosenMRIGrp 26341786 Apr 19 2020 hg19_cnvkit_filtered_ref.cnn -rwxrwxrwx 1 gailr rosenMRIGrp 3157608038 Mar 26 2017 hg19full.fa -rwxrwxrwx 1 gailr rosenMRIGrp 788 Mar 26 2017 hg19full.fa.fai -rwxrwxrwx 1 gailr rosenMRIGrp 21600 Aug 5 2020 hg19_merged_centromeres_conserved_sorted.bed -rwxrwxrwx 1 gailr rosenMRIGrp 788 Jul 23 2019 hg19_noAlt.fa.fai drwxrwsrwx 2 gailr rosenMRIGrp 90 Mar 26 2017 human_hg19_september_2011 -rwxrwxrwx 1 gailr rosenMRIGrp 1344853487 Mar 26 2017 wgEncodeDukeMapabilityUniqueness35bp_sorted.bedGraph -rwxrwxrwx 1 gailr rosenMRIGrp 35217 Jun 19 2020 wgMapabilityExcludable.bed

I(pysam) @.*** FastViFi]$ python3 run_kraken_vifi_container.py --singularity --skip-bwa-filter --input-file /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_1.fq --input-file-2 /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq --output-dir test --virus hpv --kraken-db-path /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets --vifi-viral-ref-dir /ifs/groups/rosenMRIGrp/gailr/FastViFi/viral_data --human-chr-list /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/human_chr_list.txt --vifi-human-ref-dir /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo/ > vifi_errors INFO: User not listed in /etc/subuid, trying root-mapped namespace INFO: Using fakeroot command combined with root-mapped namespace Loading database information... done. 19 sequences (0.00 Mbp) processed in 0.008s (138.1 Kseq/m, 21.80 Mbp/m). 19 sequences classified (100.00%) 0 sequences unclassified (0.00%) Command being timed: "/home/kraken2/kraken2 --use-names --report /dev/null --db /home/kraken2-db/Kraken2StandardDB_k_25_hpv_hg --threads 1 --paired --f-threshold 0.4 --keep-unmapped-reads --unmapped-threshold 0.8 --classified-out /home/output/reads_passing_kraken_first_level_for_virus_hpv#.fq /home/output/input_file_1.fq /home/output/input_file_2.fq --output /dev/null" User time (seconds): 0.01 System time (seconds): 2.99 Percent of CPU this job got: 34% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:08.74 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 4167528 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 37 Minor (reclaiming a frame) page faults: 938743 Voluntary context switches: 1163 Involuntary context switches: 8 Swaps: 0 File system inputs: 8335416 File system outputs: 16 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 Loading database information... done. 10 sequences (0.00 Mbp) processed in 0.007s (90.5 Kseq/m, 27.16 Mbp/m). 10 sequences classified (100.00%) 0 sequences unclassified (0.00%) Command being timed: "/home/kraken2/kraken2 --use-names --report /dev/null --db /home/kraken2-db/Kraken2StandardDB_k_22_hpv --threads 1 --paired --f-threshold 0.5 --unmapped-threshold 0.9 --classified-out /home/output/reads_passing_kraken_filter_for_virus_hpv#.fq /home/output/reads_passing_kraken_first_level_for_virus_hpv_1.fq /home/output/reads_passing_kraken_first_level_for_virus_hpv_2.fq --output /dev/null" User time (seconds): 0.01 System time (seconds): 0.00 Percent of CPU this job got: 22% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.09 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 9500 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 2614 Voluntary context switches: 206 Involuntary context switches: 1 Swaps: 0 File system inputs: 10824 File system outputs: 16 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [E::bwa_idx_load_from_disk] fail to locate the index files Traceback (most recent call last): File "/home/ViFi/scripts/get_trans_new.py", line 104, in bamFile = pysam.Samfile(opts.dataName[0], 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 996, in pysam.libcalignmentfile.AlignmentFile._open ValueError: file has no sequences defined (mode='rb') - is it SAM/BAM format? Consider opening with check_sq=False [E::hts_open_format] Failed to open file "/home/output/output_hpv.unknown.bam" : No such file or directory Traceback (most recent call last): File "/home/ViFi/scripts/run_hmms.py", line 115, in run_pipeline(options) File "/home/ViFi/scripts/run_hmms.py", line 37, in run_pipeline prepare_unmapped_sequences(options) File "/home/ViFi/scripts/run_hmms.py", line 96, in prepare_unmapped_sequences bam = pysam.Samfile(options.bamfile, 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/output/output_hpv.unknown.bam: No such file or directory [E::hts_open_format] Failed to open file "/home/output/output_hpv.trans.bam" : No such file or directory Traceback (most recent call last): File "/home/ViFi/scripts/merge_viral_reads.py", line 117, in transFile = pysam.Samfile(args.transName[0], 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/output/output_hpv.trans.bam: No such file or directory [E::hts_open_format] Failed to open file /home/output/output_hpv.fixed.trans.bam samtools sort: can't open "/home/output/output_hpv.fixed.trans.bam": No such file or directory samtools index: "/home/output/output_hpv.fixed.trans.cs.bam" is in a format that cannot be usefully indexed [E::hts_open_format] Failed to open file /home/output/output_hpv.viral.bam samtools sort: can't open "/home/output/output_hpv.viral.bam": No such file or directory samtools index: "/home/output/output_hpv.viral.cs.bam" is in a format that cannot be usefully indexed WARNING:root:#TIME 0.063 Unable to find reference in $AA_DATA_REPO/reference.txt. Setting to working directory. WARNING:root:#TIME 0.063 Unable to find reference in $AA_DATA_REPO/REF/file_list.txt. Setting to empty. [E::fai_build3_core] Failed to open the file /home/data_repo// WARNING:root:#TIME 0.063 Unable to open fasta file: "/home/data_repo//". Reference sequences will be set to N. WARNING:root:#TIME 0.063 Unable to open chromosome lengths file: "/home/data_repo//" WARNING:root:#TIME 0.063 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.063 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.063 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.063 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.063 interval_list: Unable to open interval file "/home/data_repo//". Traceback (most recent call last): File "/home/ViFi/scripts/cluster_trans_new.py", line 37, in bamFile = pysam.Samfile(args.dataName[0], 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 952, in pysam.libcalignmentfile.AlignmentFile._open ValueError: file does not contain alignment data Command being timed: "python /home/ViFi/scripts/run_vifi.py -f /home/output/reads_passing_kraken_filter_for_virus_hpv_1.fq -r /home/output/reads_passing_kraken_filter_for_virus_hpv_2.fq -o /home/output --virus hpv -c 1 --prefix output_hpv" User time (seconds): 0.26 System time (seconds): 0.08 Percent of CPU this job got: 29% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.16 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 20584 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 84 Minor (reclaiming a frame) page faults: 17684 Voluntary context switches: 5378 Involuntary context switches: 11 Swaps: 0 File system inputs: 24232 File system outputs: 24 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [E::hts_open_format] Failed to open file "/home/output/output_hpv.viral.bam" : No such file or directory Traceback (most recent call last): File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 491, in run_pipeline(args) File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 482, in run_pipeline bwa_filtered_fq_filename_2=bwa_filtered_fq_filename_2) File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 348, in run_kraken_vifi vifi_input_fq_1=vifi_input_fq_1) File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 380, in write_viral_reads_report viral_aligned_file = pysam.AlignmentFile(os.path.join(args.output_dir, "{}.viral.bam".format(prefix)), 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/output/output_hpv.viral.bam: No such file or directory Traceback (most recent call last): File "run_kraken_vifi_container.py", line 96, in call_fastvifi_pipeline(args) File "run_kraken_vifi_container.py", line 89, in call_fastvifi_pipeline shell_output = subprocess.check_output( File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 411, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 512, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command ' singularity run --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test:/home/input/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq:/home/input/test_reads_2.fq --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets:/home/kraken2-db --fakeroot --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo/:/home/data_repo/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/viral_data:/home/repo/data/ --bind test:/home/output/ --bind ./run_kraken_vifi_pipeline.py:/home/fastvifi/run_kraken_vifi_pipeline.py fastvifi_v1.1.sif python /home/fastvifi/run_kraken_vifi_pipeline.py --kraken-path /home/kraken2/kraken2 --vifi-path /home/ViFi/scripts/run_vifi.py --output /home/output --human-chr-list /home/data_repo/GRCh38/chrom_list.txt --kraken-db-path /home/kraken2-db --docker --virus hpv --input-file /home/input/test_reads_1.fq --input-file-2 /home/input/test_reads_2.fq --skip-bwa-filter ' returned non-zero exit status 1.

From: Sara Javadzadeh @.***> Sent: Thursday, October 12, 2023 3:28 PM To: sara-javadzadeh/FastViFi Cc: Rosen,Gail; Author Subject: Re: [sara-javadzadeh/FastViFi] Error on test code (however, using singularity) (Issue #10)

External.

Hi Gail,

Thanks for providing all the output information. And sorry for the delay.

Did you confirm that the SIF file was created? It doesn't take 10 hours for me to run, so that part is a bit unexpected.

Assuming the SIF file was correctly created and looking through the output, it looks like kraken filters are run without error with 10 reads passing the sequences. You can find the reads in reads_passing_kraken_filter_for_virus_hpv_1.fq and reads_passing_kraken_filter_for_virus_hpv_2.fq if you are interested.

The error comes from the ViFi step. It looks like the reference data is not accessed correctly. Could you please double check that the reference files are present in the data_repo file?

The AA_DATA_REPO environment variable is set in the Docker file as the default variable for hg reference data. But passing a directory to the --vifi-human-ref-dir flag should replace the default variable. Plus, binding the directories through singularity, it should be accessible through the environment.

Best, Sara

On Tue, Oct 3, 2023 at 11:02 AM Gail Rosen @.***> wrote:

for your reference, here is the full output

(pysam) @. FastViFi]$ python3 run_kraken_vifi_container.py --singularity --skip-bwa -filter --input-file /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_1.fq --input-file-2 /if s/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq --output-dir test --virus hpv --kr aken-db-path /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets --vifi-viral-ref-dir /ifs/group s/rosenMRIGrp/gailr/FastViFi/viral_data --human-chr-list /ifs/groups/rosenMRIGrp/gailr/FastViFi/tes t/human_chr_list.txt --vifi-human-ref-dir /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo > vifi_e rrors Loading database information... done. 19 sequences (0.00 Mbp) processed in 0.005s (248.7 Kseq/m, 39.27 Mbp/m). 19 sequences classified (100.00%) 0 sequences unclassified (0.00%) Command being timed: "/home/kraken2/kraken2 --use-names --report /dev/null --db /home/kraken2-d b/Kraken2StandardDB_k_25_hpv_hg --threads 1 --paired --f-threshold 0.4 --keep-unmapped-reads --unmapped -threshold 0.8 --classified-out /home/output/reads_passing_kraken_first_level_for_virus_hpv#.fq /home/o utput/input_file_1.fq /home/output/input_file_2.fq --output /dev/null" User time (seconds): 0.01 System time (seconds): 1.29 Percent of CPU this job got: 98% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.32 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 4167580 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 19 Minor (reclaiming a frame) page faults: 4558 Voluntary context switches: 145 Involuntary context switches: 60 Swaps: 0 File system inputs: 4094 File system outputs: 16 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 Loading database information... done. 10 sequences (0.00 Mbp) processed in 0.004s (144.8 Kseq/m, 43.43 Mbp/m). 10 sequences classified (100.00%) 0 sequences unclassified (0.00%) Command being timed: "/home/kraken2/kraken2 --use-names --report /dev/null --db /home/kraken2-db/Kraken2StandardDB_k_22_hpv --threads 1 --paired --f-threshold 0.5 --unmapped-threshold 0.9 --classified-out /home/output/reads_passing_kraken_filter_for_virus_hpv#.fq /home/output/reads_passing_kraken_first_level_for_virus_hpv_1.fq /home/output/reads_passing_kraken_first_level_for_virus_hpv_2.fq --output /dev/null" User time (seconds): 0.01 System time (seconds): 0.00 Percent of CPU this job got: 66% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.02 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 9556 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 2085 Voluntary context switches: 18 Involuntary context switches: 0 Swaps: 0 File system inputs: 0 File system outputs: 16 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [E::bwa_idx_load_from_disk] fail to locate the index files Traceback (most recent call last): File "/home/ViFi/scripts/get_trans_new.py", line 104, in bamFile = pysam.Samfile(opts.dataName[0], 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 996, in pysam.libcalignmentfile.AlignmentFile._open ValueError: file has no sequences defined (mode='rb') - is it SAM/BAM format? Consider opening with check_sq=False [E::hts_open_format] Failed to open file "/home/output/output_hpv.unknown.bam" : No such file or directory Traceback (most recent call last): File "/home/ViFi/scripts/run_hmms.py", line 115, in run_pipeline(options) File "/home/ViFi/scripts/run_hmms.py", line 37, in run_pipeline prepare_unmapped_sequences(options) File "/home/ViFi/scripts/run_hmms.py", line 96, in prepare_unmapped_sequences bam = pysam.Samfile(options.bamfile, 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/output/output_hpv.unknown.bam: No such file or directory [E::hts_open_format] Failed to open file "/home/output/output_hpv.trans.bam" : No such file or directory Traceback (most recent call last): File "/home/ViFi/scripts/merge_viral_reads.py", line 117, in transFile = pysam.Samfile(args.transName[0], 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/output/output_hpv.trans.bam: No such file or directory [E::hts_open_format] Failed to open file /home/output/output_hpv.fixed.trans.bam samtools sort: can't open "/home/output/output_hpv.fixed.trans.bam": No such file or directory samtools index: "/home/output/output_hpv.fixed.trans.cs.bam" is in a format that cannot be usefully indexed [E::hts_open_format] Failed to open file /home/output/output_hpv.viral.bam samtools sort: can't open "/home/output/output_hpv.viral.bam": No such file or directory samtools index: "/home/output/output_hpv.viral.cs.bam" is in a format that cannot be usefully indexed WARNING:root:#TIME 0.061 Unable to find reference in $AA_DATA_REPO/reference.txt. Setting to working directory. WARNING:root:#TIME 0.061 Unable to find reference in $AA_DATA_REPO/REF/file_list.txt. Setting to empty. [E::fai_build3_core] Failed to open the file /home/data_repo// WARNING:root:#TIME 0.061 Unable to open fasta file: "/home/data_repo//". Reference sequences will be set to N. WARNING:root:#TIME 0.061 Unable to open chromosome lengths file: "/home/data_repo//" WARNING:root:#TIME 0.061 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.061 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.061 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.061 interval_list: Unable to open interval file "/home/data_repo//". WARNING:root:#TIME 0.061 interval_list: Unable to open interval file "/home/data_repo//". Traceback (most recent call last): File "/home/ViFi/scripts/cluster_trans_new.py", line 37, in bamFile = pysam.Samfile(args.dataName[0], 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 952, in pysam.libcalignmentfile.AlignmentFile._open ValueError: file does not contain alignment data Command being timed: "python /home/ViFi/scripts/run_vifi.py -f /home/output/reads_passing_kraken_filter_for_virus_hpv_1.fq -r /home/output/reads_passing_kraken_filter_for_virus_hpv_2.fq -o /home/output --virus hpv -c 1 --prefix output_hpv" User time (seconds): 0.27 System time (seconds): 0.12 Percent of CPU this job got: 93% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.42 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 21024 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 48 Minor (reclaiming a frame) page faults: 17487 Voluntary context switches: 373 Involuntary context switches: 35 Swaps: 0 File system inputs: 10072 File system outputs: 24 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [E::hts_open_format] Failed to open file "/home/output/output_hpv.viral.bam" : No such file or directory Traceback (most recent call last): File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 491, in run_pipeline(args) File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 482, in run_pipeline bwa_filtered_fq_filename_2=bwa_filtered_fq_filename_2) File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 348, in run_kraken_vifi vifi_input_fq_1=vifi_input_fq_1) File "/home/fastvifi/run_kraken_vifi_pipeline.py", line 380, in write_viral_reads_report viral_aligned_file = pysam.AlignmentFile(os.path.join(args.output_dir, "{}.viral.bam".format(prefix)), 'rb') File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/output/output_hpv.viral.bam: No such file or directory Traceback (most recent call last): File "run_kraken_vifi_container.py", line 95, in call_fastvifi_pipeline(args) File "run_kraken_vifi_container.py", line 88, in call_fastvifi_pipeline shell_output = subprocess.check_output( File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 411, in check_output return run(popenargs, stdout=PIPE, timeout=timeout, check=True, File "/ifs/opt/python/anaconda3/lib/python3.8/subprocess.py", line 512, in run raise CalledProcessError(retcode, process.args, raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command ' singularity run --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test:/home/input/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/test/test_reads_2.fq:/home/input/test_reads_2.fq --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/kraken_datasets:/home/kraken2-db --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/data_repo:/home/data_repo/ --bind /ifs/groups/rosenMRIGrp/gailr/FastViFi/viral_data:/home/repo/data/ --bind test:/home/output/ --bind ./run_kraken_vifi_pipeline.py:/home/fastvifi/run_kraken_vifi_pipeline.py fastvifi_v1.1.sif python /home/fastvifi/run_kraken_vifi_pipeline.py --kraken-path /home/kraken2/kraken2 --vifi-path /home/ViFi/scripts/run_vifi.py --output /home/output --human-chr-list /home/data_repo/GRCh38/chrom_list.txt --kraken-db-path /home/kraken2-db --docker --virus hpv --input-file /home/input/test_reads_1.fq --input-file-2 /home/input/test_reads_2.fq --skip-bwa-filter ' returned non-zero exit status 1. (pysam) **@.*** FastViFi]$

— Reply to this email directly, view it on GitHub https://urldefense.com/v3/__https://github.com/sara-javadzadeh/FastViFi/issues/10*issuecomment-1745467395__;Iw!!Mih3wA!E2yNhpjfooMD1RN-W6BnKaLHHkoil6soEABfBTkWaaJZjK8Gd957a5RnA6fUwe2sZw_sag1rHN9ecz_rOxcNl1tPteI4RErC$, or unsubscribe https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AOGKYDSSZOMTUL2ZK6Y6HLTX5RHKXAVCNFSM6AAAAAAY46H2XWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBVGQ3DOMZZGU__;!!Mih3wA!E2yNhpjfooMD1RN-W6BnKaLHHkoil6soEABfBTkWaaJZjK8Gd957a5RnA6fUwe2sZw_sag1rHN9ecz_rOxcNl1tPtQxd362y$ . You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHubhttps://github.com/sara-javadzadeh/FastViFi/issues/10#issuecomment-1760246400, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABHZROMZOYL5VRWJPU5PNXDX7BAG3ANCNFSM6AAAAAAY46H2XU. You are receiving this because you authored the thread.Message ID: @.***>

sara-javadzadeh / FastViFi

Error on test code (however, using singularity) #10