search

wwood / CoverM

Read coverage calculator for metagenomics
GNU General Public License v3.0
297 stars 30 forks source link

ERROR coverm::bam_generator #185

Closed jamesriddellv closed 1 year ago

jamesriddellv commented 1 year ago

Hello,

I keep getting this error: [2023-09-19T18:10:45Z INFO coverm] CoverM version 0.6.1 [2023-09-19T18:10:45Z INFO coverm] Using min-covered-fraction 0% [2023-09-19T18:10:45Z INFO bird_tool_utils::external_command_checker] Found minimap2 version 2.21-r1071 [2023-09-19T18:10:46Z INFO bird_tool_utils::external_command_checker] Found samtools version 1.13 [2023-09-19T18:10:46Z INFO coverm] Not pre-generating minimap2 index [2023-09-19T18:10:46Z ERROR coverm::bam_generator] Failed to correctly find or parse BAM file at "/tmp/slurmtmp.24045041/coverm_fifo.zcfwtBseihgX/foo.pipe": unable to open SAM/BAM/CRAM file at /tmp/slurmtmp.24045041/coverm_fifo.zcfwtBseihgX/foo.pipe

I have checked some of the other issues and tried to increase my memory by using 300GB vs 177GB and I still get the same error. I also tried specifying my TMPDIR as my primary directory and get the same error.

Interestingly, this error occurs whether I supply bam files or try to use bwa within CoverM. These two scripts both give me the same error: module load singularity/current singularity run /users/PAS1117/osu9664/eMicro-Apps/CoverM-0.6.1.sif contig \ -1 $R1_loc \ -2 $R2_loc \ --reference vOTUs.sh --mapper bwa-mem --output-format sparse \ --min-read-percent-identity .95 \ --min-read-aligned-length 50 \ --min-covered-fraction 10 \ -m mean \ -t 40 > 02-coverage/${sample}_meancov.txt

#!/bin/bash [redacted] module load singularity/current singularity run /users/PAS1117/osu9664/eMicro-Apps/CoverM-0.6.1.sif \ contig \ --coupled $R1_loc $R2_loc \ --reference vOTUs.fna --output-format dense \ --min-read-percent-identity .95 \ --min-read-aligned-length 50 \ --min-covered-fraction 10 \ -m mean \ --bam-files ${WORKING_DIR}/bam-files/${sample}/${sample}.bam \ -t 40 > 02-coverage/${sample}_meancov.txt

My sorted bam file is 8GB (generated using bowtie2), my paired reads are 4GB each, my reference is 6.2GB.

I was wondering if it's possible that I needed to have an indexed bam file? Or an updated version of Samtools somehow? I know it needs to be sorted but wasn't sure if it needed to be indexed. To me it seems like it's not recognizing the location of the BAM files.

Any ideas would be super helpful, thanks!

Sincerely, James

wwood commented 1 year ago

Hi,

Strange. When you supply --bam-files it shouldn't require much RAM - I would start narrowing the problem from there. Can you provide a full stdout and stderr for that please?

It shouldn't need an index.

Thanks.

jamesriddellv commented 1 year ago

Hi Ben,

I let the program run to completion and here is the full stdout + stderr (I did not separate them).

I am also currently re-running the samtools sort step to see if my file was not sorted somehow.

`(base) [riddell26@pitzer-login01 scripts]$ cat slurm-coverm_24045862_1.out WARNING: Environment variable LD_PRELOAD already has value [], will not forward new value [/apps/xalt/xalt/lib64/libxalt_init.so] from parent process environment [2023-09-19T19:39:53Z INFO coverm] CoverM version 0.6.1 [2023-09-19T19:39:53Z INFO coverm] Using min-covered-fraction 0% [2023-09-19T19:39:53Z INFO bird_tool_utils::external_command_checker] Found minimap2 version 2.21-r1071 [2023-09-19T19:39:53Z INFO bird_tool_utils::external_command_checker] Found samtools version 1.13 [2023-09-19T19:39:53Z INFO coverm] Not pre-generating minimap2 index [2023-09-19T19:39:53Z ERROR coverm::bam_generator] Failed to correctly find or parse BAM file at "/tmp/slurmtmp.24045862/coverm_fifo.iDW6B40yngGu/foo.pipe": unable to open SAM/BAM/CRAM file at /tmp/slurmtmp.24045862/coverm_fifo.iDW6B40yngGu/foo.pipe [2023-09-19T21:21:06Z ERROR coverm::bam_generator] Error when running mapping process. Exitstatus was ExitStatus(ExitStatus(256)). Command run was: ["bash -c \"set -e -o pipefail; minimap2 --split-prefix /tmp/slurmtmp.24045862/.tmpcKar28 -a -x sr -t 1 \'/fs/ess/PAS1117/riddell26/Cyanophage_Ecogenomics/00-compile-ref-seqs/vOTU-clustering/cyanophages_u_GOV2.fna.self-blastn.clusters.fna\' \'/fs/ess/PAS1117/GOV2_Reads/TARA_READS_4_READMAPPING/Station100_SUR_R1.fastq.gz\' \'/fs/ess/PAS1117/GOV2_Reads/TARA_READS_4_READMAPPING/Station100_SUR_R2.fastq.gz\' 2>/tmp/slurmtmp.24045862/.tmpHlDSMz | remove_minimap2_duplicated_headers 2>/tmp/slurmtmp.24045862/.tmpil6giF| samtools sort -T \'/tmp/slurmtmp.24045862/coverm_fifo.iDW6B40yngGu/coverm-make-samtools-sort4gssCc\' -l0 -@ 0 2>/tmp/slurmtmp.24045862/.tmpEVGkSz > \"/tmp/slurmtmp.24045862/coverm_fifo.iDW6B40yngGu/foo.pipe\"\""] [2023-09-19T21:21:06Z ERROR coverm::bam_generator] The STDERR for the MINIMAP2_SR part was: [M::mm_idx_gen::111.7071.00] collected minimizers [M::mm_idx_gen::158.3581.00] sorted minimizers [M::main::158.3581.00] loaded/built the index for 167034 target sequence(s) [M::mm_mapopt_update::158.3581.00] mid_occ = 1000 [M::mm_idx_stat] kmer size: 21; skip: 11; is_hpc: 0; #seq: 167034 [M::mm_idx_stat::166.9621.00] distinct minimizers: 548948575 (85.24% are singletons); average occurrences: 1.212; average spacing: 6.016; total length: 4001194384 [M::worker_pipeline::184.3791.00] mapped 517118 sequences [M::worker_pipeline::200.5061.00] mapped 514990 sequences [M::worker_pipeline::216.2381.00] mapped 512984 sequences [M::worker_pipeline::231.4531.00] mapped 512486 sequences [M::worker_pipeline::246.7391.00] mapped 512174 sequences [M::worker_pipeline::261.4121.00] mapped 511980 sequences [M::worker_pipeline::276.4901.00] mapped 512174 sequences [M::worker_pipeline::291.4421.00] mapped 512422 sequences [M::worker_pipeline::306.6040.99] mapped 511948 sequences [M::worker_pipeline::321.5340.99] mapped 511890 sequences [M::worker_pipeline::336.3440.99] mapped 512514 sequences [M::worker_pipeline::350.7430.99] mapped 513408 sequences [M::worker_pipeline::365.5180.99] mapped 511890 sequences [M::worker_pipeline::379.9960.99] mapped 512436 sequences [M::worker_pipeline::394.2130.99] mapped 509606 sequences [M::worker_pipeline::408.6020.99] mapped 509224 sequences [M::worker_pipeline::423.0510.99] mapped 511664 sequences [M::worker_pipeline::437.9840.99] mapped 514912 sequences [M::worker_pipeline::452.6190.99] mapped 512860 sequences [M::worker_pipeline::466.9411.00] mapped 517932 sequences [M::worker_pipeline::481.3961.00] mapped 514802 sequences [M::worker_pipeline::495.8281.00] mapped 521378 sequences [M::worker_pipeline::510.3260.99] mapped 513010 sequences [M::worker_pipeline::525.2020.99] mapped 517262 sequences [M::worker_pipeline::539.9111.00] mapped 513110 sequences [M::worker_pipeline::554.4991.00] mapped 517628 sequences [M::worker_pipeline::569.8501.00] mapped 513318 sequences [M::worker_pipeline::584.5321.00] mapped 512258 sequences [M::worker_pipeline::599.0571.00] mapped 514282 sequences [M::worker_pipeline::613.8881.00] mapped 520058 sequences [M::worker_pipeline::628.6421.00] mapped 514874 sequences [M::worker_pipeline::642.8031.00] mapped 518110 sequences [M::worker_pipeline::657.6581.00] mapped 513708 sequences [M::worker_pipeline::672.6251.00] mapped 512630 sequences [M::worker_pipeline::689.2971.00] mapped 512734 sequences [M::worker_pipeline::705.5821.00] mapped 513028 sequences [M::worker_pipeline::721.7061.00] mapped 513490 sequences [M::worker_pipeline::737.4631.00] mapped 512684 sequences [M::worker_pipeline::752.5001.00] mapped 513616 sequences [M::worker_pipeline::767.4491.00] mapped 513350 sequences [M::worker_pipeline::782.0991.00] mapped 512574 sequences [M::worker_pipeline::796.5541.00] mapped 513944 sequences [M::worker_pipeline::810.8501.00] mapped 513532 sequences [M::worker_pipeline::825.2191.00] mapped 514162 sequences [M::worker_pipeline::840.5591.00] mapped 510130 sequences [M::worker_pipeline::855.3461.00] mapped 508638 sequences [M::worker_pipeline::869.5331.00] mapped 512148 sequences [M::worker_pipeline::883.9171.00] mapped 515334 sequences [M::worker_pipeline::898.6221.00] mapped 513094 sequences [M::worker_pipeline::912.9311.00] mapped 519644 sequences [M::worker_pipeline::927.2181.00] mapped 515728 sequences [M::worker_pipeline::942.0601.00] mapped 522984 sequences [M::worker_pipeline::956.9301.00] mapped 513656 sequences [M::worker_pipeline::971.5091.00] mapped 518364 sequences [M::worker_pipeline::986.3051.00] mapped 513700 sequences [M::worker_pipeline::1000.6461.00] mapped 519158 sequences [M::worker_pipeline::1015.2861.00] mapped 513400 sequences [M::worker_pipeline::1030.1741.00] mapped 513206 sequences [M::worker_pipeline::1044.9301.00] mapped 513532 sequences [M::worker_pipeline::1059.3451.00] mapped 516942 sequences [M::worker_pipeline::1073.6421.00] mapped 517122 sequences [M::worker_pipeline::1088.0481.00] mapped 523794 sequences [M::worker_pipeline::1102.8931.00] mapped 516682 sequences [M::worker_pipeline::1117.4591.00] mapped 516104 sequences [M::worker_pipeline::1131.7851.00] mapped 515950 sequences [M::worker_pipeline::1146.1191.00] mapped 515710 sequences [M::worker_pipeline::1161.0811.00] mapped 515710 sequences [M::worker_pipeline::1175.2501.00] mapped 515820 sequences [M::worker_pipeline::1189.5691.00] mapped 516370 sequences [M::worker_pipeline::1203.9371.00] mapped 515510 sequences [M::worker_pipeline::1219.3821.00] mapped 515446 sequences [M::worker_pipeline::1233.7341.00] mapped 514876 sequences [M::worker_pipeline::1247.9781.00] mapped 517006 sequences [M::worker_pipeline::1262.4611.00] mapped 514932 sequences [M::worker_pipeline::1277.2391.00] mapped 515460 sequences [M::worker_pipeline::1291.6661.00] mapped 512298 sequences [M::worker_pipeline::1306.0631.00] mapped 512620 sequences [M::worker_pipeline::1320.4321.00] mapped 513486 sequences [M::worker_pipeline::1334.6681.00] mapped 516986 sequences [M::worker_pipeline::1349.5331.00] mapped 514534 sequences [M::worker_pipeline::1363.8941.00] mapped 520864 sequences [M::worker_pipeline::1379.0291.00] mapped 518412 sequences [M::worker_pipeline::1393.2971.00] mapped 525966 sequences [M::worker_pipeline::1407.6471.00] mapped 515906 sequences [M::worker_pipeline::1421.7131.00] mapped 521526 sequences [M::worker_pipeline::1436.5801.00] mapped 515448 sequences [M::worker_pipeline::1450.7661.00] mapped 520874 sequences [M::worker_pipeline::1465.5341.00] mapped 515820 sequences [M::worker_pipeline::1479.9241.00] mapped 515498 sequences [M::worker_pipeline::1494.0951.00] mapped 515866 sequences [M::worker_pipeline::1508.2751.00] mapped 521044 sequences [M::worker_pipeline::1522.8011.00] mapped 517080 sequences [M::worker_pipeline::1538.1501.00] mapped 518932 sequences [M::worker_pipeline::1552.3751.00] mapped 513376 sequences [M::worker_pipeline::1566.8391.00] mapped 512942 sequences [M::worker_pipeline::1581.2361.00] mapped 512590 sequences [M::worker_pipeline::1596.0171.00] mapped 512518 sequences [M::worker_pipeline::1610.7451.00] mapped 512734 sequences [M::worker_pipeline::1625.6741.00] mapped 512484 sequences [M::worker_pipeline::1639.9801.00] mapped 513376 sequences [M::worker_pipeline::1654.9301.00] mapped 512700 sequences [M::worker_pipeline::1669.6641.00] mapped 512264 sequences [M::worker_pipeline::1683.9331.00] mapped 513538 sequences [M::worker_pipeline::1698.9241.00] mapped 513760 sequences [M::worker_pipeline::1713.8661.00] mapped 512042 sequences [M::worker_pipeline::1728.4371.00] mapped 511050 sequences [M::worker_pipeline::1742.5131.00] mapped 509020 sequences [M::worker_pipeline::1756.8531.00] mapped 512044 sequences [M::worker_pipeline::1771.2151.00] mapped 516402 sequences [M::worker_pipeline::1786.2121.00] mapped 513830 sequences [M::worker_pipeline::1801.5031.00] mapped 517990 sequences [M::worker_pipeline::1816.8411.00] mapped 515886 sequences [M::worker_pipeline::1831.9151.00] mapped 525150 sequences [M::worker_pipeline::1846.6781.00] mapped 513712 sequences [M::worker_pipeline::1860.8281.00] mapped 518622 sequences [M::worker_pipeline::1875.5491.00] mapped 513636 sequences [M::worker_pipeline::1890.4191.00] mapped 518718 sequences [M::worker_pipeline::1904.9121.00] mapped 515346 sequences [M::worker_pipeline::1919.4201.00] mapped 514420 sequences [M::worker_pipeline::1935.0611.00] mapped 515996 sequences [M::worker_pipeline::1951.1601.00] mapped 521940 sequences [M::worker_pipeline::1965.3471.00] mapped 515882 sequences [M::worker_pipeline::1979.8641.00] mapped 517294 sequences [M::worker_pipeline::1994.0211.00] mapped 512832 sequences [M::worker_pipeline::2008.6291.00] mapped 512280 sequences [M::worker_pipeline::2023.2471.00] mapped 511488 sequences [M::worker_pipeline::2037.8761.00] mapped 512338 sequences [M::worker_pipeline::2052.3201.00] mapped 512578 sequences [M::worker_pipeline::2066.6951.00] mapped 511828 sequences [M::worker_pipeline::2081.5611.00] mapped 512598 sequences [M::worker_pipeline::2095.9481.00] mapped 512552 sequences [M::worker_pipeline::2110.7641.00] mapped 511706 sequences [M::worker_pipeline::2126.1211.00] mapped 512850 sequences [M::worker_pipeline::2140.8121.00] mapped 512746 sequences [M::worker_pipeline::2155.7261.00] mapped 511880 sequences [M::worker_pipeline::2170.4671.00] mapped 509282 sequences [M::worker_pipeline::2184.9541.00] mapped 508860 sequences [M::worker_pipeline::2199.7541.00] mapped 512222 sequences [M::worker_pipeline::2214.2951.00] mapped 515072 sequences [M::worker_pipeline::2228.7961.00] mapped 512780 sequences [M::worker_pipeline::2242.8311.00] mapped 517758 sequences [M::worker_pipeline::2257.9361.00] mapped 514868 sequences [M::worker_pipeline::2272.1111.00] mapped 521288 sequences [M::worker_pipeline::2287.4661.00] mapped 512992 sequences [M::worker_pipeline::2301.2441.00] mapped 518180 sequences [M::worker_pipeline::2315.6521.00] mapped 513256 sequences [M::worker_pipeline::2330.3821.00] mapped 516708 sequences [M::worker_pipeline::2345.2881.00] mapped 513900 sequences [M::worker_pipeline::2360.1651.00] mapped 513856 sequences [M::worker_pipeline::2374.8041.00] mapped 514220 sequences [M::worker_pipeline::2389.7011.00] mapped 520508 sequences [M::worker_pipeline::2405.0661.00] mapped 516440 sequences [M::worker_pipeline::2420.1801.00] mapped 521590 sequences [M::worker_pipeline::2434.7041.00] mapped 515518 sequences [M::worker_pipeline::2449.3791.00] mapped 515516 sequences [M::worker_pipeline::2464.5131.00] mapped 514626 sequences [M::worker_pipeline::2481.5171.00] mapped 514572 sequences [M::worker_pipeline::2495.7641.00] mapped 514762 sequences [M::worker_pipeline::2510.6701.00] mapped 514288 sequences [M::worker_pipeline::2525.3721.00] mapped 514512 sequences [M::worker_pipeline::2540.5681.00] mapped 513768 sequences [M::worker_pipeline::2554.7701.00] mapped 513532 sequences [M::worker_pipeline::2570.0381.00] mapped 513536 sequences [M::worker_pipeline::2584.8701.00] mapped 516164 sequences [M::worker_pipeline::2599.6201.00] mapped 514382 sequences [M::worker_pipeline::2613.8571.00] mapped 514732 sequences [M::worker_pipeline::2628.8771.00] mapped 512668 sequences [M::worker_pipeline::2643.7771.00] mapped 510994 sequences [M::worker_pipeline::2658.2781.00] mapped 513024 sequences [M::worker_pipeline::2672.5731.00] mapped 516404 sequences [M::worker_pipeline::2687.1641.00] mapped 513970 sequences [M::worker_pipeline::2701.9821.00] mapped 518406 sequences [M::worker_pipeline::2717.7331.00] mapped 517942 sequences [M::worker_pipeline::2732.4651.00] mapped 526052 sequences [M::worker_pipeline::2747.3681.00] mapped 515368 sequences [M::worker_pipeline::2762.2671.00] mapped 521488 sequences [M::worker_pipeline::2777.4241.00] mapped 515284 sequences [M::worker_pipeline::2791.6271.00] mapped 520306 sequences [M::worker_pipeline::2806.9831.00] mapped 516946 sequences [M::worker_pipeline::2821.3531.00] mapped 514846 sequences [M::worker_pipeline::2835.7941.00] mapped 514664 sequences [M::worker_pipeline::2850.2341.00] mapped 516480 sequences [M::worker_pipeline::2851.8121.00] mapped 58594 sequences [M::mm_idx_gen::2922.1011.00] collected minimizers [M::mm_idx_gen::2953.4441.00] sorted minimizers [M::main::2953.4451.00] loaded/built the index for 348327 target sequence(s) [M::mm_mapopt_update::2953.4451.00] mid_occ = 1000 [M::mm_idx_stat] kmer size: 21; skip: 11; is_hpc: 0; #seq: 348327 [M::mm_idx_stat::2959.8581.00] distinct minimizers: 342685144 (85.87% are singletons); average occurrences: 1.208; average spacing: 6.040; total length: 2500804476 [M::worker_pipeline::2978.4651.00] mapped 517118 sequences [M::worker_pipeline::2998.4801.00] mapped 514990 sequences [M::worker_pipeline::3018.1131.00] mapped 512984 sequences [M::worker_pipeline::3037.5091.00] mapped 512486 sequences [M::worker_pipeline::3056.5211.00] mapped 512174 sequences [M::worker_pipeline::3074.1771.00] mapped 511980 sequences [M::worker_pipeline::3092.7201.00] mapped 512174 sequences [M::worker_pipeline::3111.9241.00] mapped 512422 sequences [M::worker_pipeline::3129.5291.00] mapped 511948 sequences [M::worker_pipeline::3148.8301.00] mapped 511890 sequences [M::worker_pipeline::3167.3121.00] mapped 512514 sequences [M::worker_pipeline::3184.5491.00] mapped 513408 sequences [M::worker_pipeline::3202.8481.00] mapped 511890 sequences [M::worker_pipeline::3219.7891.00] mapped 512436 sequences [M::worker_pipeline::3238.3011.00] mapped 509606 sequences [M::worker_pipeline::3255.8361.00] mapped 509224 sequences [M::worker_pipeline::3273.3491.00] mapped 511664 sequences [M::worker_pipeline::3289.8931.00] mapped 514912 sequences [M::worker_pipeline::3307.2181.00] mapped 512860 sequences [M::worker_pipeline::3323.5351.00] mapped 517932 sequences [M::worker_pipeline::3340.3821.00] mapped 514802 sequences [M::worker_pipeline::3356.7791.00] mapped 521378 sequences [M::worker_pipeline::3374.1511.00] mapped 513010 sequences [M::worker_pipeline::3391.2061.00] mapped 517262 sequences [M::worker_pipeline::3408.0231.00] mapped 513110 sequences [M::worker_pipeline::3424.8891.00] mapped 517628 sequences [M::worker_pipeline::3442.1401.00] mapped 513318 sequences [M::worker_pipeline::3459.3021.00] mapped 512258 sequences [M::worker_pipeline::3475.8691.00] mapped 514282 sequences [M::worker_pipeline::3492.8451.00] mapped 520058 sequences [M::worker_pipeline::3509.6811.00] mapped 514874 sequences [M::worker_pipeline::3526.3841.00] mapped 518110 sequences [M::worker_pipeline::3542.6971.00] mapped 513708 sequences [M::worker_pipeline::3559.5041.00] mapped 512630 sequences [M::worker_pipeline::3577.1341.00] mapped 512734 sequences [M::worker_pipeline::3595.6161.00] mapped 513028 sequences [M::worker_pipeline::3613.6431.00] mapped 513490 sequences [M::worker_pipeline::3630.8931.00] mapped 512684 sequences [M::worker_pipeline::3647.7641.00] mapped 513616 sequences [M::worker_pipeline::3664.7351.00] mapped 513350 sequences [M::worker_pipeline::3681.3541.00] mapped 512574 sequences [M::worker_pipeline::3697.7121.00] mapped 513944 sequences [M::worker_pipeline::3714.6341.00] mapped 513532 sequences [M::worker_pipeline::3730.7331.00] mapped 514162 sequences [M::worker_pipeline::3748.1231.00] mapped 510130 sequences [M::worker_pipeline::3764.8171.00] mapped 508638 sequences [M::worker_pipeline::3781.7131.00] mapped 512148 sequences [M::worker_pipeline::3798.0301.00] mapped 515334 sequences [M::worker_pipeline::3815.2991.00] mapped 513094 sequences [M::worker_pipeline::3831.2681.00] mapped 519644 sequences [M::worker_pipeline::3848.2771.00] mapped 515728 sequences [M::worker_pipeline::3864.4891.00] mapped 522984 sequences [M::worker_pipeline::3882.7011.00] mapped 513656 sequences [M::worker_pipeline::3901.4811.00] mapped 518364 sequences [M::worker_pipeline::3920.0261.00] mapped 513700 sequences [M::worker_pipeline::3937.2731.00] mapped 519158 sequences [M::worker_pipeline::3954.4061.00] mapped 513400 sequences [M::worker_pipeline::3970.9711.00] mapped 513206 sequences [M::worker_pipeline::3988.2341.00] mapped 513532 sequences [M::worker_pipeline::4004.6201.00] mapped 516942 sequences [M::worker_pipeline::4021.3291.00] mapped 517122 sequences [M::worker_pipeline::4037.6151.00] mapped 523794 sequences [M::worker_pipeline::4054.3581.00] mapped 516682 sequences [M::worker_pipeline::4070.9721.00] mapped 516104 sequences [M::worker_pipeline::4087.5551.00] mapped 515950 sequences [M::worker_pipeline::4104.2501.00] mapped 515710 sequences [M::worker_pipeline::4121.2381.00] mapped 515710 sequences [M::worker_pipeline::4137.8661.00] mapped 515820 sequences [M::worker_pipeline::4154.2501.00] mapped 516370 sequences [M::worker_pipeline::4170.9021.00] mapped 515510 sequences [M::worker_pipeline::4187.7871.00] mapped 515446 sequences [M::worker_pipeline::4205.2081.00] mapped 514876 sequences [M::worker_pipeline::4221.3531.00] mapped 517006 sequences [M::worker_pipeline::4238.6831.00] mapped 514932 sequences [M::worker_pipeline::4255.2681.00] mapped 515460 sequences [M::worker_pipeline::4272.1531.00] mapped 512298 sequences [M::worker_pipeline::4288.5171.00] mapped 512620 sequences [M::worker_pipeline::4305.4071.00] mapped 513486 sequences [M::worker_pipeline::4321.4991.00] mapped 516986 sequences [M::worker_pipeline::4338.2541.00] mapped 514534 sequences [M::worker_pipeline::4354.7951.00] mapped 520864 sequences [M::worker_pipeline::4371.8501.00] mapped 518412 sequences [M::worker_pipeline::4388.1631.00] mapped 525966 sequences [M::worker_pipeline::4405.1231.00] mapped 515906 sequences [M::worker_pipeline::4421.2191.00] mapped 521526 sequences [M::worker_pipeline::4438.6191.00] mapped 515448 sequences [M::worker_pipeline::4454.9651.00] mapped 520874 sequences [M::worker_pipeline::4471.8561.00] mapped 515820 sequences [M::worker_pipeline::4488.5031.00] mapped 515498 sequences [M::worker_pipeline::4505.0171.00] mapped 515866 sequences [M::worker_pipeline::4521.5591.00] mapped 521044 sequences [M::worker_pipeline::4538.0241.00] mapped 517080 sequences [M::worker_pipeline::4555.5301.00] mapped 518932 sequences [M::worker_pipeline::4572.0921.00] mapped 513376 sequences [M::worker_pipeline::4588.9911.00] mapped 512942 sequences [M::worker_pipeline::4605.4061.00] mapped 512590 sequences [M::worker_pipeline::4622.6931.00] mapped 512518 sequences [M::worker_pipeline::4639.4911.00] mapped 512734 sequences [M::worker_pipeline::4657.0381.00] mapped 512484 sequences [M::worker_pipeline::4673.3381.00] mapped 513376 sequences [M::worker_pipeline::4690.6971.00] mapped 512700 sequences [M::worker_pipeline::4707.0541.00] mapped 512264 sequences [M::worker_pipeline::4723.9851.00] mapped 513538 sequences [M::worker_pipeline::4740.9911.00] mapped 513760 sequences [M::worker_pipeline::4758.4861.00] mapped 512042 sequences [M::worker_pipeline::4774.9581.00] mapped 511050 sequences [M::worker_pipeline::4791.8331.00] mapped 509020 sequences [M::worker_pipeline::4808.1481.00] mapped 512044 sequences [M::worker_pipeline::4825.1601.00] mapped 516402 sequences [M::worker_pipeline::4841.7321.00] mapped 513830 sequences [M::worker_pipeline::4859.6391.00] mapped 517990 sequences [M::worker_pipeline::4876.6531.00] mapped 515886 sequences [M::worker_pipeline::4894.1551.00] mapped 525150 sequences [M::worker_pipeline::4910.7901.00] mapped 513712 sequences [M::worker_pipeline::4927.5031.00] mapped 518622 sequences [M::worker_pipeline::4944.2861.00] mapped 513636 sequences [M::worker_pipeline::4961.6221.00] mapped 518718 sequences [M::worker_pipeline::4977.9651.00] mapped 515346 sequences [M::worker_pipeline::4994.9381.00] mapped 514420 sequences [M::worker_pipeline::5012.5851.00] mapped 515996 sequences [M::worker_pipeline::5030.6861.00] mapped 521940 sequences [M::worker_pipeline::5046.8401.00] mapped 515882 sequences [M::worker_pipeline::5063.6641.00] mapped 517294 sequences [M::worker_pipeline::5080.1551.00] mapped 512832 sequences [M::worker_pipeline::5097.2091.00] mapped 512280 sequences [M::worker_pipeline::5113.7941.00] mapped 511488 sequences [M::worker_pipeline::5130.8431.00] mapped 512338 sequences [M::worker_pipeline::5147.6671.00] mapped 512578 sequences [M::worker_pipeline::5164.5101.00] mapped 511828 sequences [M::worker_pipeline::5181.4631.00] mapped 512598 sequences [M::worker_pipeline::5198.1461.00] mapped 512552 sequences [M::worker_pipeline::5215.6861.00] mapped 511706 sequences [M::worker_pipeline::5233.2341.00] mapped 512850 sequences [M::worker_pipeline::5250.0751.00] mapped 512746 sequences [M::worker_pipeline::5267.2951.00] mapped 511880 sequences [M::worker_pipeline::5284.4701.00] mapped 509282 sequences [M::worker_pipeline::5301.4601.00] mapped 508860 sequences [M::worker_pipeline::5318.4541.00] mapped 512222 sequences [M::worker_pipeline::5335.2611.00] mapped 515072 sequences [M::worker_pipeline::5352.1421.00] mapped 512780 sequences [M::worker_pipeline::5368.5481.00] mapped 517758 sequences [M::worker_pipeline::5385.8981.00] mapped 514868 sequences [M::worker_pipeline::5402.2441.00] mapped 521288 sequences [M::worker_pipeline::5419.8641.00] mapped 512992 sequences [M::worker_pipeline::5435.8991.00] mapped 518180 sequences [M::worker_pipeline::5452.6531.00] mapped 513256 sequences [M::worker_pipeline::5469.4581.00] mapped 516708 sequences [M::worker_pipeline::5486.7491.00] mapped 513900 sequences [M::worker_pipeline::5503.5091.00] mapped 513856 sequences [M::worker_pipeline::5520.6781.00] mapped 514220 sequences [M::worker_pipeline::5537.4421.00] mapped 520508 sequences [M::worker_pipeline::5554.6871.00] mapped 516440 sequences [M::worker_pipeline::5571.1751.00] mapped 521590 sequences [M::worker_pipeline::5588.1871.00] mapped 515518 sequences [M::worker_pipeline::5604.8401.00] mapped 515516 sequences [M::worker_pipeline::5622.4151.00] mapped 514626 sequences [M::worker_pipeline::5641.7491.00] mapped 514572 sequences [M::worker_pipeline::5658.4731.00] mapped 514762 sequences [M::worker_pipeline::5675.4541.00] mapped 514288 sequences [M::worker_pipeline::5692.4151.00] mapped 514512 sequences [M::worker_pipeline::5709.6901.00] mapped 513768 sequences [M::worker_pipeline::5726.4891.00] mapped 513532 sequences [M::worker_pipeline::5743.7531.00] mapped 513536 sequences [M::worker_pipeline::5760.9531.00] mapped 516164 sequences [M::worker_pipeline::5777.5881.00] mapped 514382 sequences [M::worker_pipeline::5794.4931.00] mapped 514732 sequences [M::worker_pipeline::5811.5861.00] mapped 512668 sequences [M::worker_pipeline::5828.8651.00] mapped 510994 sequences [M::worker_pipeline::5845.3841.00] mapped 513024 sequences [M::worker_pipeline::5862.2621.00] mapped 516404 sequences [M::worker_pipeline::5879.2251.00] mapped 513970 sequences [M::worker_pipeline::5896.5541.00] mapped 518406 sequences [M::worker_pipeline::5914.4101.00] mapped 517942 sequences [M::worker_pipeline::5932.0081.00] mapped 526052 sequences [M::worker_pipeline::5948.8561.00] mapped 515368 sequences [M::worker_pipeline::5965.8711.00] mapped 521488 sequences [M::worker_pipeline::5983.3721.00] mapped 515284 sequences [M::worker_pipeline::6000.3581.00] mapped 520306 sequences [M::worker_pipeline::6018.3161.00] mapped 516946 sequences [M::worker_pipeline::6035.0601.00] mapped 514846 sequences [M::worker_pipeline::6051.7311.00] mapped 514664 sequences [M::worker_pipeline::6068.2831.00] mapped 516480 sequences [M::worker_pipeline::6070.1021.00] mapped 58594 sequences

[2023-09-19T21:21:06Z ERROR coverm::bam_generator] The STDERR for the samtools sort part was: samtools sort: failed to read header from "-"

[2023-09-19T21:21:06Z ERROR coverm::bam_generator] The STDERR for the remove_minimap2_duplicated_headers part was: bash: remove_minimap2_duplicated_headers: command not found

[2023-09-19T21:21:06Z ERROR coverm::bam_generator] Cannot continue since mapping failed. /var/spool/slurmd/job24045862/slurm_script: line 30: 02-coverage/Station100_SUR_meancov.txt: No such file or directory`

jamesriddellv commented 1 year ago

EDIT: I see that issue 59 had the same problem: https://github.com/wwood/CoverM/issues/59. I will ask about getting a fresh install of coverM to fix the issue, but it still may not explain why it is unable to read the BAM files.

Sorry didn't mean to close it. Is it somehow not recognizing the location of my bam files?

jamesriddellv commented 1 year ago

Update: I realized in my coverm runscript I missed a slash after --reference so the command was not seeing all of the parameters being passed.

I got this new error:

`(base) [riddell26@pitzer-login01 scripts]$ cat slurm-coverm_24053366_1.err WARNING: Environment variable LD_PRELOAD already has value [], will not forward new value [/apps/xalt/xalt/lib64/libxalt_init.so] from parent process environment error: The argument '--bam-files ...' cannot be used with '--coupled ...'

USAGE: coverm contig --bam-files ... --contig-end-exclusion --coupled ... --interleaved ... --mapper --methods ... --min-covered-fraction --min-read-aligned-length --min-read-percent-identity --output-format -1 ... -2 ... --reference ... --single ... --threads --trim-max --trim-min

For more information try --help`

I did not realize --bam-files meant that it no longer needed --reference or either of the read pairs. I guess it makes sense now when I think about it since it just needs the sorted bams to calculate the coverage, and the other parts are only needed for the read mapping. The whole program ran in a few seconds on the login node.

To summarize:

  1. Missing '\' character preventing the program from reading more of the lines (I should have thought to look for these yesterday but I had my blinders on...oops!)
  2. Didn't need the reference or read pairs since bam files were already made.