search

weng-lab / TEMP2

Algorithm to detect germline and de novo transposon insertions
https://publications.wenglab.org/TEMP2/
GNU General Public License v3.0
26 stars 6 forks source link

Fail to run TEMP2 #22

Closed Song-10-YF closed 9 months ago

Song-10-YF commented 10 months ago

Dear Dr. Yu When running TEMP2 with a BAM file generated from my own mapping, I encountered a problem. It's important to note that running the test data went very smoothly.

My command TEMP2 insertion -l _1.clean.fq.gz -r _2.clean.fq.gz -i Cyc-BFYt1.realigned.bam -g reference_diploid.fa -R TEib.fasta -t TE.bed -o out -c 20 -T

log Testing required softwares: bwa: /home/songyf/anaconda3/bin/bwa samtools: /home/songyf/anaconda3/bin/samtools bedtools: /home/songyf/anaconda3/bin/bedtools ------ Start pipeline ------ get concordant-uniq-split reads 2024年 02月 02日 星期五 08:44:02 CST [bam_sort_core] merging from 0 files and 80 in-memory blocks... [M::bam2fq_mainloop] discarded 0 singletons [M::bam2fq_mainloop] processed 1513961 reads check fragment length 2024年 02月 02日 星期五 09:54:57 CST insert size set to 95 quantile: 441 get mate seq of the uniq-unpaired 2024年 02月 02日 星期五 09:54:59 CST [M::bam2fq_mainloop] discarded 0 singletons [M::bam2fq_mainloop] processed 0 reads map paired split uniqMappers and unpaired uniqMappers to transposons 2024年 02月 02日 星期五 09:56:00 CST merge fragments in genome and transposon 2024年 02月 02日 星期五 09:56:21 CST pass1 - making usageList (16 chroms): 3 millis pass2 - checking and writing primary data (6860 records, 6 fields): 30 millis merge support reads in the same direction within 441 - 2024年 02月 02日 星期五 09:56:41 CST expr: 语法错误 /home/songyf/Software/TEMP2-master/bin/TEMP2_insertion.sh: 第 208 行:[: -lt: 期待一元表达式

Tool: bedtools merge (aka mergeBed) Version: v2.31.1 Summary: Merges overlapping BED/GFF/VCF entries into a single interval.

Usage: bedtools merge [OPTIONS] -i <bed/gff/vcf>

Options: -s Force strandedness. That is, only merge features that are on the same strand.

Notes: (1) The input file (-i) file must be sorted by chrom, then start.

ERROR: -d option must be followed by an integer value

Tool: bedtools merge (aka mergeBed) Version: v2.31.1 Summary: Merges overlapping BED/GFF/VCF entries into a single interval.

Usage: bedtools merge [OPTIONS] -i <bed/gff/vcf>

Options: -s Force strandedness. That is, only merge features that are on the same strand.

Notes: (1) The input file (-i) file must be sorted by chrom, then start.

ERROR: -d option must be followed by an integer value Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range merge support reads in different direction within 2 X 441 - 2024年 02月 02日 星期五 09:56:55 CST filter candidate insertions which overlap with the same transposon insertion or in high depth region 2024年 02月 02日 星期五 09:56:56 CST filter candidate insertions in high depth region 2024年 02月 02日 星期五 09:56:56 CST [W::hts_idx_load3] The index file is older than the data file: /home/songyf/Juglandaceae/Cpa_TEploy/Cyc-WYS77.sorted.bam.bai average read number for 200bp bins is 207.195, set read number cutoff to 1035.97 Filtered insertion number: 0 - 0 (overlap rmsk) 0 (short insertion) - 0 (high depth) = 0

***** ERROR: Requested column 4, but database file - only has fields 1 - 0. generate the overall distribution of transposon mapping reads, first map all reads to transposon2024年 02月 02日 星期五 10:17:06 CST sam to bed and bedGraph, multiple mappers are divided by their map times 2024年 02月 02日 星期五 10:26:14 CST [bam_sort_core] merging from 0 files and 80 in-memory blocks... estimate de novo insertion number for each transposon using singleton reads 2024年 02月 02日 星期五 10:27:51 CST generate distribution figures for singleton supporting reads 2024年 02月 02日 星期五 10:27:54 CST Error in read.table(Args[8], header = F, row.names = NULL) : 输入中没有多出的行 停止执行 filter unreliable singleton insertions, also filter 2p insertions overlapped with similar reference transposon copies 2024年 02月 02日 星期五 10:27:55 CST Calculate frequency of each transposon insertion 2024年 02月 02日 星期五 10:27:55 CST [bam_sort_core] merging from 2 files and 80 in-memory blocks... get TSD, remove redundant insertions and recalculate de novo insertion rate 2024年 02月 02日 星期五 11:08:08 CST

***** ERROR: Requested column 2, but database file - only has fields 1 - 0. Cyc-BFYt1_1.clean.fq.gz.t is empty calculate de novo insertion rate per genome 2024年 02月 02日 星期五 11:08:13 CST clean tmp files 2024年 02月 02日 星期五 11:08:13 CST Done, Congras!!!🍺🍺🍺

BAM header @HD VN:1.5 SO:coordinate @SQ SN:Contig01 LN:30214 @SQ SN:Contig02 LN:125000 @SQ SN:Contig03 LN:47009 @SQ SN:Contig04 LN:38938 @SQ SN:Contig05 LN:38666 @SQ SN:Contig06 LN:35601 @SQ SN:Contig07 LN:28829 @SQ SN:Contig08 LN:26945 @SQ SN:Contig09 LN:26821 @SQ SN:Contig10 LN:25243 @SQ SN:Contig11 LN:25072 @SQ SN:Contig12 LN:19593 @SQ SN:Contig13 LN:17151 @SQ SN:Contig14 LN:47820 @SQ SN:Contig15 LN:33168 @SQ SN:Contig16 LN:23323 @SQ SN:Contig17 LN:29869 @SQ SN:DCPChr01 LN:45280996 @SQ SN:DCPChr02 LN:41078791 @SQ SN:DCPChr03 LN:37187694 @SQ SN:DCPChr04 LN:36825483 @SQ SN:DCPChr05 LN:29600334 @SQ SN:DCPChr06 LN:41820140 @SQ SN:DCPChr07 LN:61288576 @SQ SN:DCPChr08 LN:34313777 @SQ SN:DCPChr09 LN:25093348 @SQ SN:DCPChr10 LN:38614602 @SQ SN:DCPChr11 LN:38760090 @SQ SN:DCPChr12 LN:34697173 @SQ SN:DCPChr13 LN:45502784 @SQ SN:DCPChr14 LN:33169711 @SQ SN:DCPChr15 LN:25614943 @SQ SN:DCPChr16 LN:31997599 @RG ID:Cyc-BFYt1 SM:Cyc-BFYt1 PL:Illumina @PG ID:bwa CL:bwa mem -M -R @RG\tID:Cyc-BFYt1\tSM:Cyc-BFYt1\tPL:Illumina -t 60 ./reference/reference_diploid.fa ./trim/Cyc-BFYt1.trim1.fq.gz ./trim/Cyc-BFYt1.trim2.fq.gz PN:bwa VN:0.7.17-r1188 @PG ID:samtools.1 CL:samtools view -bS Cyc-BFYt1.f3.sam PN:samtools PP:samtools VN:1.17 @PG ID:samtools CL:samtools view -f 3 -F 2304 -h -S Cyc-BFYt1.sam PN:samtools PP:bwa VN:1.17 @PG ID:sentieon-Dedup CL:/panfs3/software/sentieon-genomics-202308/libexec/driver -t 30 -i Cyc-BFYt1.sorted.bam --algo LocusCollector --fun score_info Cyc-BFYt1.score.txt;/panfs3/software/sentieon-genomics-202308/libexec/driver -t 30 -i Cyc-BFYt1.sorted.bam --algo Dedup --rmdup --score_info Cyc-BFYt1.score.txt --metrics Cyc-BFYt1.dedup_metrics.txt Cyc-BFYt1.deduped.bam PN:sentieon-Dedup PP:sentieon-sort VN:sentieon-genomics-202308 @PG ID:sentieon-sort CL:/panfs3/software/sentieon-genomics-202308/libexec/util sort -o Cyc-BFYt1.sorted.bam -t 30 -i Cyc-BFYt1.uniq.bam PN:sentieon-sort PP:samtools.1 VN:sentieon-genomics-202308 @PG ID:sentieon-Realigner CL:/panfs3/software/sentieon-genomics-202308/libexec/driver -r ./reference/reference_diploid.fa -t 30 -i Cyc-BFYt1.deduped.bam --algo Realigner Cyc-BFYt1.realigned.bam PN:sentieon-Realigner PP:sentieon-Dedup VN:sentieon-genomics-202308 @PG ID:samtools.2 PN:samtools PP:sentieon-Realigner VN:1.19.1 CL:samtools view -H Cyc-BFYt1.realigned.bam

I suspect that Sentieon has affected the BAM files, so I only re-sorted the original BAM and encountered the same error. I would like to inquire about the reason for this. I look forward to your reply. Best wishes. Yanfeng

tianxiongbb commented 10 months ago

Hi Yanfeng,

TEMP2 uses raw bwa mapping file for TE insertion detection, so any filtering by sam flags are not suggested before running TEMP2.

Best, Tianxiong

On Feb 2, 2024, at 11:20 PM, Yan-Feng Song @.***> wrote:

Dear Dr. Yu When running TEMP2 with a BAM file generated from my own mapping, I encountered a problem. It's important to note that running the test data went very smoothly.

My command TEMP2 insertion -l _1.clean.fq.gz -r _2.clean.fq.gz -i Cyc-BFYt1.realigned.bam -g reference_diploid.fa -R TEib.fasta -t TE.bed -o out -c 20 -T

log Testing required softwares: bwa: /home/songyf/anaconda3/bin/bwa samtools: /home/songyf/anaconda3/bin/samtools bedtools: /home/songyf/anaconda3/bin/bedtools ------ Start pipeline ------ get concordant-uniq-split reads 2024年 02月 02日 星期五 08:44:02 CST [bam_sort_core] merging from 0 files and 80 in-memory blocks... [M::bam2fq_mainloop] discarded 0 singletons [M::bam2fq_mainloop] processed 1513961 reads check fragment length 2024年 02月 02日 星期五 09:54:57 CST insert size set to 95 quantile: 441 get mate seq of the uniq-unpaired 2024年 02月 02日 星期五 09:54:59 CST [M::bam2fq_mainloop] discarded 0 singletons [M::bam2fq_mainloop] processed 0 reads map paired split uniqMappers and unpaired uniqMappers to transposons 2024年 02月 02日 星期五 09:56:00 CST merge fragments in genome and transposon 2024年 02月 02日 星期五 09:56:21 CST pass1 - making usageList (16 chroms): 3 millis pass2 - checking and writing primary data (6860 records, 6 fields): 30 millis merge support reads in the same direction within 441 - 2024年 02月 02日 星期五 09:56:41 CST expr: 语法错误 /home/songyf/Software/TEMP2-master/bin/TEMP2_insertion.sh: 第 208 行:[: -lt: 期待一元表达式

Tool: bedtools merge (aka mergeBed) Version: v2.31.1 Summary: Merges overlapping BED/GFF/VCF entries into a single interval.

Usage: bedtools merge [OPTIONS] -i <bed/gff/vcf>

Options: -s Force strandedness. That is, only merge features that are on the same strand.

  • By default, merging is done without respect to strand.

-S Force merge for one specific strand only. Follow with + or - to force merge from only the forward or reverse strand, respectively.

  • By default, merging is done without respect to strand.

-d Maximum distance between features allowed for features to be merged.

  • Def. 0. That is, overlapping & book-ended features are merged.
  • (INTEGER)
  • Note: negative values enforce the number of b.p. required for overlap.

-c Specify columns from the B file to map onto intervals in A. Default: 5. Multiple columns can be specified in a comma-delimited list.

-o Specify the operation that should be applied to -c. Valid operations: sum, min, max, absmin, absmax, mean, median, mode, antimode stdev, sstdev collapse (i.e., print a delimited list (duplicates allowed)), distinct (i.e., print a delimited list (NO duplicates allowed)), distinct_sort_num (as distinct, sorted numerically, ascending), distinct_sort_num_desc (as distinct, sorted numerically, desscending), distinct_only (delimited list of only unique values), count count_distinct (i.e., a count of the unique values in the column), first (i.e., just the first value in the column), last (i.e., just the last value in the column), Default: sum Multiple operations can be specified in a comma-delimited list.

If there is only column, but multiple operations, all operations will be applied on that column. Likewise, if there is only one operation, but multiple columns, that operation will be applied to all columns. Otherwise, the number of columns must match the the number of operations, and will be applied in respective order. E.g., "-c 5,4,6 -o sum,mean,count" will give the sum of column 5, the mean of column 4, and the count of column 6. The order of output columns will match the ordering given in the command.

-delim Specify a custom delimiter for the collapse operations.

  • Example: -delim "|"
  • Default: ",".

-prec Sets the decimal precision for output (Default: 5)

-bed If using BAM input, write output as BED.

-header Print the header from the A file prior to results.

-nobuf Disable buffered output. Using this option will cause each line of output to be printed as it is generated, rather than saved in a buffer. This will make printing large output files noticeably slower, but can be useful in conjunction with other software tools and scripts that need to process one line of bedtools output at a time.

-iobuf Specify amount of memory to use for input buffer. Takes an integer argument. Optional suffixes K/M/G supported. Note: currently has no effect with compressed files. Notes: (1) The input file (-i) file must be sorted by chrom, then start.

ERROR: -d option must be followed by an integer value

Tool: bedtools merge (aka mergeBed) Version: v2.31.1 Summary: Merges overlapping BED/GFF/VCF entries into a single interval.

Usage: bedtools merge [OPTIONS] -i <bed/gff/vcf>

Options: -s Force strandedness. That is, only merge features that are on the same strand.

  • By default, merging is done without respect to strand.

-S Force merge for one specific strand only. Follow with + or - to force merge from only the forward or reverse strand, respectively.

  • By default, merging is done without respect to strand.

-d Maximum distance between features allowed for features to be merged.

  • Def. 0. That is, overlapping & book-ended features are merged.
  • (INTEGER)
  • Note: negative values enforce the number of b.p. required for overlap.

-c Specify columns from the B file to map onto intervals in A. Default: 5. Multiple columns can be specified in a comma-delimited list.

-o Specify the operation that should be applied to -c. Valid operations: sum, min, max, absmin, absmax, mean, median, mode, antimode stdev, sstdev collapse (i.e., print a delimited list (duplicates allowed)), distinct (i.e., print a delimited list (NO duplicates allowed)), distinct_sort_num (as distinct, sorted numerically, ascending), distinct_sort_num_desc (as distinct, sorted numerically, desscending), distinct_only (delimited list of only unique values), count count_distinct (i.e., a count of the unique values in the column), first (i.e., just the first value in the column), last (i.e., just the last value in the column), Default: sum Multiple operations can be specified in a comma-delimited list.

If there is only column, but multiple operations, all operations will be applied on that column. Likewise, if there is only one operation, but multiple columns, that operation will be applied to all columns. Otherwise, the number of columns must match the the number of operations, and will be applied in respective order. E.g., "-c 5,4,6 -o sum,mean,count" will give the sum of column 5, the mean of column 4, and the count of column 6. The order of output columns will match the ordering given in the command.

-delim Specify a custom delimiter for the collapse operations.

  • Example: -delim "|"
  • Default: ",".

-prec Sets the decimal precision for output (Default: 5)

-bed If using BAM input, write output as BED.

-header Print the header from the A file prior to results.

-nobuf Disable buffered output. Using this option will cause each line of output to be printed as it is generated, rather than saved in a buffer. This will make printing large output files noticeably slower, but can be useful in conjunction with other software tools and scripts that need to process one line of bedtools output at a time.

-iobuf Specify amount of memory to use for input buffer. Takes an integer argument. Optional suffixes K/M/G supported. Note: currently has no effect with compressed files. Notes: (1) The input file (-i) file must be sorted by chrom, then start.

ERROR: -d option must be followed by an integer value Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range Traceback (most recent call last): File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in main() File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main ins = int(sys.argv[3]) IndexError: list index out of range merge support reads in different direction within 2 X 441 - 2024年 02月 02日 星期五 09:56:55 CST filter candidate insertions which overlap with the same transposon insertion or in high depth region 2024年 02月 02日 星期五 09:56:56 CST filter candidate insertions in high depth region 2024年 02月 02日 星期五 09:56:56 CST [W::hts_idx_load3] The index file is older than the data file: /home/songyf/Juglandaceae/Cpa_TEploy/Cyc-WYS77.sorted.bam.bai average read number for 200bp bins is 207.195, set read number cutoff to 1035.97 Filtered insertion number: 0 - 0 (overlap rmsk) 0 (short insertion) - 0 (high depth) = 0

***** ERROR: Requested column 4, but database file - only has fields 1 - 0. generate the overall distribution of transposon mapping reads, first map all reads to transposon2024年 02月 02日 星期五 10:17:06 CST sam to bed and bedGraph, multiple mappers are divided by their map times 2024年 02月 02日 星期五 10:26:14 CST [bam_sort_core] merging from 0 files and 80 in-memory blocks... estimate de novo insertion number for each transposon using singleton reads 2024年 02月 02日 星期五 10:27:51 CST generate distribution figures for singleton supporting reads 2024年 02月 02日 星期五 10:27:54 CST Error in read.table(Args[8], header = F, row.names = NULL) : 输入中没有多出的行 停止执行 filter unreliable singleton insertions, also filter 2p insertions overlapped with similar reference transposon copies 2024年 02月 02日 星期五 10:27:55 CST Calculate frequency of each transposon insertion 2024年 02月 02日 星期五 10:27:55 CST [bam_sort_core] merging from 2 files and 80 in-memory blocks... get TSD, remove redundant insertions and recalculate de novo insertion rate 2024年 02月 02日 星期五 11:08:08 CST

***** ERROR: Requested column 2, but database file - only has fields 1 - 0. Cyc-BFYt1_1.clean.fq.gz.t is empty calculate de novo insertion rate per genome 2024年 02月 02日 星期五 11:08:13 CST clean tmp files 2024年 02月 02日 星期五 11:08:13 CST Done, Congras!!!🍺🍺🍺

BAM header @hd https://github.com/hd VN:1.5 SO:coordinate @sq https://github.com/sq SN:Contig01 LN:30214 @sq https://github.com/sq SN:Contig02 LN:125000 @sq https://github.com/sq SN:Contig03 LN:47009 @sq https://github.com/sq SN:Contig04 LN:38938 @sq https://github.com/sq SN:Contig05 LN:38666 @sq https://github.com/sq SN:Contig06 LN:35601 @sq https://github.com/sq SN:Contig07 LN:28829 @sq https://github.com/sq SN:Contig08 LN:26945 @sq https://github.com/sq SN:Contig09 LN:26821 @sq https://github.com/sq SN:Contig10 LN:25243 @sq https://github.com/sq SN:Contig11 LN:25072 @sq https://github.com/sq SN:Contig12 LN:19593 @sq https://github.com/sq SN:Contig13 LN:17151 @sq https://github.com/sq SN:Contig14 LN:47820 @sq https://github.com/sq SN:Contig15 LN:33168 @sq https://github.com/sq SN:Contig16 LN:23323 @sq https://github.com/sq SN:Contig17 LN:29869 @sq https://github.com/sq SN:DCPChr01 LN:45280996 @sq https://github.com/sq SN:DCPChr02 LN:41078791 @sq https://github.com/sq SN:DCPChr03 LN:37187694 @sq https://github.com/sq SN:DCPChr04 LN:36825483 @sq https://github.com/sq SN:DCPChr05 LN:29600334 @sq https://github.com/sq SN:DCPChr06 LN:41820140 @sq https://github.com/sq SN:DCPChr07 LN:61288576 @sq https://github.com/sq SN:DCPChr08 LN:34313777 @sq https://github.com/sq SN:DCPChr09 LN:25093348 @sq https://github.com/sq SN:DCPChr10 LN:38614602 @sq https://github.com/sq SN:DCPChr11 LN:38760090 @sq https://github.com/sq SN:DCPChr12 LN:34697173 @sq https://github.com/sq SN:DCPChr13 LN:45502784 @sq https://github.com/sq SN:DCPChr14 LN:33169711 @sq https://github.com/sq SN:DCPChr15 LN:25614943 @sq https://github.com/sq SN:DCPChr16 LN:31997599 @rg https://github.com/rg ID:Cyc-BFYt1 SM:Cyc-BFYt1 PL:Illumina @pg https://github.com/pg ID:bwa CL:bwa mem -M -R @rg https://github.com/rg\tID:Cyc-BFYt1\tSM:Cyc-BFYt1\tPL:Illumina -t 60 ./reference/reference_diploid.fa ./trim/Cyc-BFYt1.trim1.fq.gz ./trim/Cyc-BFYt1.trim2.fq.gz PN:bwa VN:0.7.17-r1188 @pg https://github.com/pg ID:samtools.1 CL:samtools view -bS Cyc-BFYt1.f3.sam PN:samtools PP:samtools VN:1.17 @pg https://github.com/pg ID:samtools CL:samtools view -f 3 -F 2304 -h -S Cyc-BFYt1.sam PN:samtools PP:bwa VN:1.17 @pg https://github.com/pg ID:sentieon-Dedup CL:/panfs3/software/sentieon-genomics-202308/libexec/driver -t 30 -i Cyc-BFYt1.sorted.bam --algo LocusCollector --fun score_info Cyc-BFYt1.score.txt;/panfs3/software/sentieon-genomics-202308/libexec/driver -t 30 -i Cyc-BFYt1.sorted.bam --algo Dedup --rmdup --score_info Cyc-BFYt1.score.txt --metrics Cyc-BFYt1.dedup_metrics.txt Cyc-BFYt1.deduped.bam PN:sentieon-Dedup PP:sentieon-sort VN:sentieon-genomics-202308 @pg https://github.com/pg ID:sentieon-sort CL:/panfs3/software/sentieon-genomics-202308/libexec/util sort -o Cyc-BFYt1.sorted.bam -t 30 -i Cyc-BFYt1.uniq.bam PN:sentieon-sort PP:samtools.1 VN:sentieon-genomics-202308 @pg https://github.com/pg ID:sentieon-Realigner CL:/panfs3/software/sentieon-genomics-202308/libexec/driver -r ./reference/reference_diploid.fa -t 30 -i Cyc-BFYt1.deduped.bam --algo Realigner Cyc-BFYt1.realigned.bam PN:sentieon-Realigner PP:sentieon-Dedup VN:sentieon-genomics-202308 @pg https://github.com/pg ID:samtools.2 PN:samtools PP:sentieon-Realigner VN:1.19.1 CL:samtools view -H Cyc-BFYt1.realigned.bam

I suspect that Sentieon has affected the BAM files, so I only re-sorted the original BAM and encountered the same error. I would like to inquire about the reason for this. I look forward to your reply. Best wishes. Yanfeng

— Reply to this email directly, view it on GitHub https://github.com/weng-lab/TEMP2/issues/22, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFPDEDD7DNOYUJLTTGGPE3LYRW3IJAVCNFSM6AAAAABCXVIJGOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGEYTMMRQGQZTANY. You are receiving this because you are subscribed to this thread.

Song-10-YF commented 10 months ago

Thank you very much!

------------------ Original ------------------ From: Tianxiong @.> Date: Mon,Feb 5,2024 5:06 AM To: weng-lab/TEMP2 @.> Cc: Yan-Feng Song @.>, Author @.> Subject: Re: [weng-lab/TEMP2] Fail to run TEMP2 (Issue #22)

Hi Yanfeng,

TEMP2 uses raw bwa mapping file for TE insertion detection, so any filtering by sam flags are not suggested before running TEMP2.

Best, Tianxiong

> On Feb 2, 2024, at 11:20 PM, Yan-Feng Song @.> wrote: >
>
> Dear Dr. Yu > When running TEMP2 with a BAM file generated from my own mapping, I encountered a problem. It's important to note that running the test data went very smoothly. >
> My command > TEMP2 insertion -l _1.clean.fq.gz -r *_2.clean.fq.gz -i Cyc-BFYt1.realigned.bam -g reference_diploid.fa -R TEib.fasta -t TE.bed -o out -c 20 -T >
> log > Testing required softwares: > bwa: /home/songyf/anaconda3/bin/bwa > samtools: /home/songyf/anaconda3/bin/samtools > bedtools: /home/songyf/anaconda3/bin/bedtools > ------ Start pipeline ------ > get concordant-uniq-split reads 2024年 02月 02日 星期五 08:44:02 CST > [bam_sort_core] merging from 0 files and 80 in-memory blocks... > [M::bam2fq_mainloop] discarded 0 singletons > [M::bam2fq_mainloop] processed 1513961 reads > check fragment length 2024年 02月 02日 星期五 09:54:57 CST > insert size set to 95 quantile: 441 > get mate seq of the uniq-unpaired 2024年 02月 02日 星期五 09:54:59 CST > [M::bam2fq_mainloop] discarded 0 singletons > [M::bam2fq_mainloop] processed 0 reads > map paired split uniqMappers and unpaired uniqMappers to transposons 2024年 02月 02日 星期五 09:56:00 CST > merge fragments in genome and transposon 2024年 02月 02日 星期五 09:56:21 CST > pass1 - making usageList (16 chroms): 3 millis > pass2 - checking and writing primary data (6860 records, 6 fields): 30 millis > merge support reads in the same direction within 441 - 2024年 02月 02日 星期五 09:56:41 CST > expr: 语法错误 > /home/songyf/Software/TEMP2-master/bin/TEMP2_insertion.sh: 第 208 行:[: -lt: 期待一元表达式 >
> Tool: bedtools merge (aka mergeBed) > Version: v2.31.1 > Summary: Merges overlapping BED/GFF/VCF entries into a single interval. >
> Usage: bedtools merge [OPTIONS] -i <bed/gff/vcf> >
> Options: > -s Force strandedness. That is, only merge features > that are on the same strand. > - By default, merging is done without respect to strand. >
> -S Force merge for one specific strand only. > Follow with + or - to force merge from only > the forward or reverse strand, respectively. > - By default, merging is done without respect to strand. >
> -d Maximum distance between features allowed for features > to be merged. > - Def. 0. That is, overlapping & book-ended features are merged. > - (INTEGER) > - Note: negative values enforce the number of b.p. required for overlap. >
> -c Specify columns from the B file to map onto intervals in A. > Default: 5. > Multiple columns can be specified in a comma-delimited list. >
> -o Specify the operation that should be applied to -c. > Valid operations: > sum, min, max, absmin, absmax, > mean, median, mode, antimode > stdev, sstdev > collapse (i.e., print a delimited list (duplicates allowed)),
> distinct (i.e., print a delimited list (NO duplicates allowed)),
> distinct_sort_num (as distinct, sorted numerically, ascending), > distinct_sort_num_desc (as distinct, sorted numerically, desscending), > distinct_only (delimited list of only unique values), > count > count_distinct (i.e., a count of the unique values in the column),
> first (i.e., just the first value in the column),
> last (i.e., just the last value in the column),
> Default: sum > Multiple operations can be specified in a comma-delimited list. >
> If there is only column, but multiple operations, all operations will be > applied on that column. Likewise, if there is only one operation, but > multiple columns, that operation will be applied to all columns. > Otherwise, the number of columns must match the the number of operations, > and will be applied in respective order. > E.g., "-c 5,4,6 -o sum,mean,count" will give the sum of column 5, > the mean of column 4, and the count of column 6. > The order of output columns will match the ordering given in the command. >
>
> -delim Specify a custom delimiter for the collapse operations. > - Example: -delim "|" > - Default: ",". >
> -prec Sets the decimal precision for output (Default: 5) >
> -bed If using BAM input, write output as BED. >
> -header Print the header from the A file prior to results. >
> -nobuf Disable buffered output. Using this option will cause each line > of output to be printed as it is generated, rather than saved > in a buffer. This will make printing large output files
> noticeably slower, but can be useful in conjunction with > other software tools and scripts that need to process one > line of bedtools output at a time. >
> -iobuf Specify amount of memory to use for input buffer. > Takes an integer argument. Optional suffixes K/M/G supported. > Note: currently has no effect with compressed files. > Notes: > (1) The input file (-i) file must be sorted by chrom, then start. >
> *** ERROR: -d option must be followed by an integer value >
> Tool: bedtools merge (aka mergeBed) > Version: v2.31.1 > Summary: Merges overlapping BED/GFF/VCF entries into a single interval. >
> Usage: bedtools merge [OPTIONS] -i <bed/gff/vcf> >
> Options: > -s Force strandedness. That is, only merge features > that are on the same strand. > - By default, merging is done without respect to strand. >
> -S Force merge for one specific strand only. > Follow with + or - to force merge from only > the forward or reverse strand, respectively. > - By default, merging is done without respect to strand. >
> -d Maximum distance between features allowed for features > to be merged. > - Def. 0. That is, overlapping & book-ended features are merged. > - (INTEGER) > - Note: negative values enforce the number of b.p. required for overlap. >
> -c Specify columns from the B file to map onto intervals in A. > Default: 5. > Multiple columns can be specified in a comma-delimited list. >
> -o Specify the operation that should be applied to -c. > Valid operations: > sum, min, max, absmin, absmax, > mean, median, mode, antimode > stdev, sstdev > collapse (i.e., print a delimited list (duplicates allowed)),
> distinct (i.e., print a delimited list (NO duplicates allowed)),
> distinct_sort_num (as distinct, sorted numerically, ascending), > distinct_sort_num_desc (as distinct, sorted numerically, desscending), > distinct_only (delimited list of only unique values), > count > count_distinct (i.e., a count of the unique values in the column),
> first (i.e., just the first value in the column),
> last (i.e., just the last value in the column),
> Default: sum > Multiple operations can be specified in a comma-delimited list. >
> If there is only column, but multiple operations, all operations will be > applied on that column. Likewise, if there is only one operation, but > multiple columns, that operation will be applied to all columns. > Otherwise, the number of columns must match the the number of operations, > and will be applied in respective order. > E.g., "-c 5,4,6 -o sum,mean,count" will give the sum of column 5, > the mean of column 4, and the count of column 6. > The order of output columns will match the ordering given in the command. >
>
> -delim Specify a custom delimiter for the collapse operations. > - Example: -delim "|" > - Default: ",". >
> -prec Sets the decimal precision for output (Default: 5) >
> -bed If using BAM input, write output as BED. >
> -header Print the header from the A file prior to results. >
> -nobuf Disable buffered output. Using this option will cause each line > of output to be printed as it is generated, rather than saved > in a buffer. This will make printing large output files
> noticeably slower, but can be useful in conjunction with > other software tools and scripts that need to process one > line of bedtools output at a time. >
> -iobuf Specify amount of memory to use for input buffer. > Takes an integer argument. Optional suffixes K/M/G supported. > Note: currently has no effect with compressed files. > Notes: > (1) The input file (-i) file must be sorted by chrom, then start. >
> ERROR: -d option must be followed by an integer value * > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > Traceback (most recent call last): > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 108, in
> main() > File "/home/songyf/Software/TEMP2-master/bin/processMergedBed.py", line 11, in main > ins = int(sys.argv[3]) > IndexError: list index out of range > merge support reads in different direction within 2 X 441 - 2024年 02月 02日 星期五 09:56:55 CST > filter candidate insertions which overlap with the same transposon insertion or in high depth region 2024年 02月 02日 星期五 09:56:56 CST > filter candidate insertions in high depth region 2024年 02月 02日 星期五 09:56:56 CST > [W::hts_idx_load3] The index file is older than the data file: /home/songyf/Juglandaceae/Cpa_TEploy/Cyc-WYS77.sorted.bam.bai > average read number for 200bp bins is 207.195, set read number cutoff to 1035.97 > Filtered insertion number: 0 - 0 (overlap rmsk) 0 (short insertion) - 0 (high depth) = 0 >
> * ERROR: Requested column 4, but database file - only has fields 1 - 0. > generate the overall distribution of transposon mapping reads, first map all reads to transposon2024年 02月 02日 星期五 10:17:06 CST > sam to bed and bedGraph, multiple mappers are divided by their map times 2024年 02月 02日 星期五 10:26:14 CST > [bam_sort_core] merging from 0 files and 80 in-memory blocks... > estimate de novo insertion number for each transposon using singleton reads 2024年 02月 02日 星期五 10:27:51 CST > generate distribution figures for singleton supporting reads 2024年 02月 02日 星期五 10:27:54 CST > Error in read.table(Args[8], header = F, row.names = NULL) : > 输入中没有多出的行 > 停止执行 > filter unreliable singleton insertions, also filter 2p insertions overlapped with similar reference transposon copies 2024年 02月 02日 星期五 10:27:55 CST > Calculate frequency of each transposon insertion 2024年 02月 02日 星期五 10:27:55 CST > [bam_sort_core] merging from 2 files and 80 in-memory blocks... > get TSD, remove redundant insertions and recalculate de novo insertion rate 2024年 02月 02日 星期五 11:08:08 CST >
> ***** ERROR: Requested column 2, but database file - only has fields 1 - 0. > Cyc-BFYt1_1.clean.fq.gz.t is empty > calculate de novo insertion rate per genome 2024年 02月 02日 星期五 11:08:13 CST > clean tmp files 2024年 02月 02日 星期五 11:08:13 CST > Done, Congras!!!🍺🍺🍺 >
> BAM header > @hd <https://github.com/hd&gt; VN:1.5 SO:coordinate > @sq <https://github.com/sq&gt; SN:Contig01 LN:30214 > @sq <https://github.com/sq&gt; SN:Contig02 LN:125000 > @sq <https://github.com/sq&gt; SN:Contig03 LN:47009 > @sq <https://github.com/sq&gt; SN:Contig04 LN:38938 > @sq <https://github.com/sq&gt; SN:Contig05 LN:38666 > @sq <https://github.com/sq&gt; SN:Contig06 LN:35601 > @sq <https://github.com/sq&gt; SN:Contig07 LN:28829 > @sq <https://github.com/sq&gt; SN:Contig08 LN:26945 > @sq <https://github.com/sq&gt; SN:Contig09 LN:26821 > @sq <https://github.com/sq&gt; SN:Contig10 LN:25243 > @sq <https://github.com/sq&gt; SN:Contig11 LN:25072 > @sq <https://github.com/sq&gt; SN:Contig12 LN:19593 > @sq <https://github.com/sq&gt; SN:Contig13 LN:17151 > @sq <https://github.com/sq&gt; SN:Contig14 LN:47820 > @sq <https://github.com/sq&gt; SN:Contig15 LN:33168 > @sq <https://github.com/sq&gt; SN:Contig16 LN:23323 > @sq <https://github.com/sq&gt; SN:Contig17 LN:29869 > @sq <https://github.com/sq&gt; SN:DCPChr01 LN:45280996 > @sq <https://github.com/sq&gt; SN:DCPChr02 LN:41078791 > @sq <https://github.com/sq&gt; SN:DCPChr03 LN:37187694 > @sq <https://github.com/sq&gt; SN:DCPChr04 LN:36825483 > @sq <https://github.com/sq&gt; SN:DCPChr05 LN:29600334 > @sq <https://github.com/sq&gt; SN:DCPChr06 LN:41820140 > @sq <https://github.com/sq&gt; SN:DCPChr07 LN:61288576 > @sq <https://github.com/sq&gt; SN:DCPChr08 LN:34313777 > @sq <https://github.com/sq&gt; SN:DCPChr09 LN:25093348 > @sq <https://github.com/sq&gt; SN:DCPChr10 LN:38614602 > @sq <https://github.com/sq&gt; SN:DCPChr11 LN:38760090 > @sq <https://github.com/sq&gt; SN:DCPChr12 LN:34697173 > @sq <https://github.com/sq&gt; SN:DCPChr13 LN:45502784 > @sq <https://github.com/sq&gt; SN:DCPChr14 LN:33169711 > @sq <https://github.com/sq&gt; SN:DCPChr15 LN:25614943 > @sq <https://github.com/sq&gt; SN:DCPChr16 LN:31997599 > @rg <https://github.com/rg&gt; ID:Cyc-BFYt1 SM:Cyc-BFYt1 PL:Illumina > @pg <https://github.com/pg&gt; ID:bwa CL:bwa mem -M -R @rg <https://github.com/rg&gt;\tID:Cyc-BFYt1\tSM:Cyc-BFYt1\tPL:Illumina -t 60 ./reference/reference_diploid.fa ./trim/Cyc-BFYt1.trim1.fq.gz ./trim/Cyc-BFYt1.trim2.fq.gz PN:bwa VN:0.7.17-r1188 > @pg <https://github.com/pg&gt; ID:samtools.1 CL:samtools view -bS Cyc-BFYt1.f3.sam PN:samtools PP:samtools VN:1.17 > @pg <https://github.com/pg&gt; ID:samtools CL:samtools view -f 3 -F 2304 -h -S Cyc-BFYt1.sam PN:samtools PP:bwa VN:1.17 > @pg <https://github.com/pg&gt; ID:sentieon-Dedup CL:/panfs3/software/sentieon-genomics-202308/libexec/driver -t 30 -i Cyc-BFYt1.sorted.bam --algo LocusCollector --fun score_info Cyc-BFYt1.score.txt;/panfs3/software/sentieon-genomics-202308/libexec/driver -t 30 -i Cyc-BFYt1.sorted.bam --algo Dedup --rmdup --score_info Cyc-BFYt1.score.txt --metrics Cyc-BFYt1.dedup_metrics.txt Cyc-BFYt1.deduped.bam PN:sentieon-Dedup PP:sentieon-sort VN:sentieon-genomics-202308 > @pg <https://github.com/pg&gt; ID:sentieon-sort CL:/panfs3/software/sentieon-genomics-202308/libexec/util sort -o Cyc-BFYt1.sorted.bam -t 30 -i Cyc-BFYt1.uniq.bam PN:sentieon-sort PP:samtools.1 VN:sentieon-genomics-202308 > @pg <https://github.com/pg&gt; ID:sentieon-Realigner CL:/panfs3/software/sentieon-genomics-202308/libexec/driver -r ./reference/reference_diploid.fa -t 30 -i Cyc-BFYt1.deduped.bam --algo Realigner Cyc-BFYt1.realigned.bam PN:sentieon-Realigner PP:sentieon-Dedup VN:sentieon-genomics-202308 > @pg <https://github.com/pg&gt; ID:samtools.2 PN:samtools PP:sentieon-Realigner VN:1.19.1 CL:samtools view -H Cyc-BFYt1.realigned.bam >
> I suspect that Sentieon has affected the BAM files, so I only re-sorted the original BAM and encountered the same error. I would like to inquire about the reason for this. I look forward to your reply. > Best wishes. > Yanfeng >
> — > Reply to this email directly, view it on GitHub <https://github.com/weng-lab/TEMP2/issues/22&gt;, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFPDEDD7DNOYUJLTTGGPE3LYRW3IJAVCNFSM6AAAAABCXVIJGOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGEYTMMRQGQZTANY&gt;. > You are receiving this because you are subscribed to this thread. >

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>