search

PacificBiosciences / kineticsTools

Tools for detecting DNA modifications from single molecule, real-time sequencing data
19 stars 21 forks source link

Stop at the same region of genome #86

Open DustinChen1986 opened 3 years ago

DustinChen1986 commented 3 years ago

Hi, I got bam with pbmm2 v1.3.0. pbmm2 align ref.mmi mymovies.fofn pbmm2.bam --sort Then I used kineticsTools: ipdSummary pbmm2.bam --reference genome.fasta --identify m6A,m4C --methylFraction --gff kinetics.gff --csv kinetics.csv --bigwig kinetics.bw --log-level INFO --numWorkers 12 --referenceStride 100000 ipdSummary started with:

[INFO] 2021-04-23 06:34:05,878Z [kineticsTools.ipdSummary _pacbio_main_runner 160] Using pbcommand v2.2.2 [INFO] 2021-04-23 06:34:05,878Z [kineticsTools.ipdSummary _pacbio_main_runner 161] completed setting up logger with <function setup_log at 0x7fd03a301ee0> [INFO] 2021-04-23 06:34:05,878Z [kineticsTools.ipdSummary _pacbio_main_runner 164] log opts {'level': 20, 'file_name': None} [INFO] 2021-04-23 06:34:05,881Z [root loadSharedAlignmentSet 475] Reading AlignmentSet: pbmm2.bam [INFO] 2021-04-23 06:34:05,881Z [root loadSharedAlignmentSet 476] reference: /mnt/hgfs/G/1.Pacbio/genome.fasta [WARNING] 2021-04-23 06:34:09,505Z [pbcore.io.dataset.DataSetIO _openFiles 2162] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality [WARNING] 2021-04-23 06:34:09,860Z [pbcore.io.dataset.DataSetIO _openFiles 2162] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality [INFO] 2021-04-23 06:34:09,987Z [kineticsTools.loader getIpdModelFilename 42] Using chemistry-matched kinetics model: '/usr/local/lib/python3.8/dist-packages/kineticsTools-0.7.0-py3.8-linux-x86_64.egg/kineticsTools/resources/SP2-C2.npz.gz' [INFO] 2021-04-23 06:34:09,987Z [root loadReferenceAndModel 463] Loading reference contigs '/mnt/hgfs/G/1.Pacbio/genome.fasta' [INFO] 2021-04-23 06:34:24,769Z [root _launchSlaveProcesses 410] Available CPUs: 12 [INFO] 2021-04-23 06:34:24,770Z [root _launchSlaveProcesses 411] Requested worker processes: 12 [INFO] 2021-04-23 06:34:24,819Z [root _run 90] Worker KineticWorkerProcess-1 (PID=97866) started running [INFO] 2021-04-23 06:34:24,826Z [root _run 90] Worker KineticWorkerProcess-2 (PID=97867) started running [INFO] 2021-04-23 06:34:24,846Z [root _run 90] Worker KineticWorkerProcess-3 (PID=97868) started running [INFO] 2021-04-23 06:34:24,850Z [root _run 90] Worker KineticWorkerProcess-5 (PID=97870) started running [INFO] 2021-04-23 06:34:24,854Z [root _run 90] Worker KineticWorkerProcess-6 (PID=97871) started running [INFO] 2021-04-23 06:34:24,858Z [root _run 90] Worker KineticWorkerProcess-4 (PID=97869) started running [INFO] 2021-04-23 06:34:24,865Z [root _launchSlaveProcesses 437] Launched worker processes. [INFO] 2021-04-23 06:34:24,866Z [root _run 90] Worker KineticWorkerProcess-7 (PID=97872) started running [INFO] 2021-04-23 06:34:24,871Z [root _launchSlaveProcesses 443] Launched result collector process. [INFO] 2021-04-23 06:34:24,873Z [root _run 90] Worker KineticWorkerProcess-8 (PID=97873) started running [INFO] 2021-04-23 06:34:24,873Z [root _run 90] Worker KineticWorkerProcess-9 (PID=97874) started running [INFO] 2021-04-23 06:34:24,880Z [kineticsTools.ResultWriter _run 40] Process KineticsWriter-13 (PID=97878) started running [INFO] 2021-04-23 06:34:24,882Z [root _mainLoop 540] Generating kinetics summary for [pbmm2.bam] [INFO] 2021-04-23 06:34:24,883Z [root _mainLoop 555] Processing window/contig: ReferenceWindow(refId=0, refName='Chr1', start=0, end=44929686) [INFO] 2021-04-23 06:34:24,886Z [root _run 90] Worker KineticWorkerProcess-12 (PID=97877) started running [INFO] 2021-04-23 06:34:24,892Z [root _run 90] Worker KineticWorkerProcess-10 (PID=97875) started running [INFO] 2021-04-23 06:34:24,899Z [root _run 90] Worker KineticWorkerProcess-11 (PID=97876) started running [INFO] 2021-04-23 06:34:26,389Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality [INFO] 2021-04-23 06:34:26,410Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality [INFO] 2021-04-23 06:34:26,413Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality [INFO] 2021-04-23 06:34:26,426Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality [INFO] 2021-04-23 06:34:26,443Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality [INFO] 2021-04-23 06:34:26,458Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality [INFO] 2021-04-23 06:34:26,467Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality [INFO] 2021-04-23 06:34:26,468Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality [INFO] 2021-04-23 06:34:26,511Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality [INFO] 2021-04-23 06:34:26,551Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality [INFO] 2021-04-23 06:34:26,599Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality [INFO] 2021-04-23 06:34:26,619Z [root _openFiles 36] pbi file missing for /mnt/hgfs/G/1.Pacbio/pbmm2.bam, operating with reduced speed and functionality [INFO] 2021-04-23 06:34:27,174Z [root _run 125] Got chunk: (0, ReferenceWindow(refId=0, refName='Chr1', start=0, end=100000)) -- Process: [INFO] 2021-04-23 06:34:27,418Z [root _run 125] Got chunk: (1, ReferenceWindow(refId=0, refName='Chr1', start=100000, end=200000)) -- Process: [INFO] 2021-04-23 06:34:27,426Z [root _run 125] Got chunk: (2, ReferenceWindow(refId=0, refName='Chr1', start=200000, end=300000)) -- Process: [INFO] 2021-04-23 06:34:27,450Z [root _run 125] Got chunk: (3, ReferenceWindow(refId=0, refName='Chr1', start=300000, end=400000)) -- Process: [INFO] 2021-04-23 06:34:27,476Z [root _run 125] Got chunk: (6, ReferenceWindow(refId=0, refName='Chr1', start=600000, end=700000)) -- Process: [INFO] 2021-04-23 06:34:27,476Z [root _run 125] Got chunk: (5, ReferenceWindow(refId=0, refName='Chr1', start=500000, end=600000)) -- Process: [INFO] 2021-04-23 06:34:27,479Z [root _run 125] Got chunk: (7, ReferenceWindow(refId=0, refName='Chr1', start=700000, end=800000)) -- Process: [INFO] 2021-04-23 06:34:27,483Z [root _run 125] Got chunk: (4, ReferenceWindow(refId=0, refName='Chr1', start=400000, end=500000)) -- Process: [INFO] 2021-04-23 06:34:28,286Z [root _run 125] Got chunk: (8, ReferenceWindow(refId=0, refName='Chr1', start=800000, end=900000)) -- Process: [INFO] 2021-04-23 06:34:28,659Z [root _run 125] Got chunk: (9, ReferenceWindow(refId=0, refName='Chr1', start=900000, end=1000000)) -- Process: [INFO] 2021-04-23 06:34:28,862Z [root _summarizeReferenceRegion 226] Making summary: -15 to 100015 [INFO] 2021-04-23 06:34:29,190Z [root _summarizeReferenceRegion 226] Making summary: 799985 to 900015 [INFO] 2021-04-23 06:34:29,642Z [root _summarizeReferenceRegion 226] Making summary: 299985 to 400015 [INFO] 2021-04-23 06:34:29,717Z [root _summarizeReferenceRegion 226] Making summary: 99985 to 200015 [INFO] 2021-04-23 06:34:29,781Z [root _summarizeReferenceRegion 226] Making summary: 599985 to 700015 [INFO] 2021-04-23 06:34:29,835Z [root _run 125] Got chunk: (10, ReferenceWindow(refId=0, refName='Chr1', start=1000000, end=1100000)) -- Process: [INFO] 2021-04-23 06:34:29,942Z [root _summarizeReferenceRegion 226] Making summary: 499985 to 600015 [INFO] 2021-04-23 06:34:29,978Z [root _run 125] Got chunk: (11, ReferenceWindow(refId=0, refName='Chr1', start=1100000, end=1200000)) -- Process: [INFO] 2021-04-23 06:34:30,005Z [root _summarizeReferenceRegion 226] Making summary: 699985 to 800015 [INFO] 2021-04-23 06:34:30,088Z [root _summarizeReferenceRegion 226] Making summary: 399985 to 500015 [INFO] 2021-04-23 06:34:30,353Z [root _summarizeReferenceRegion 226] Making summary: 199985 to 300015 [INFO] 2021-04-23 06:34:31,961Z [root _summarizeReferenceRegion 226] Making summary: 899985 to 1000015 [INFO] 2021-04-23 06:34:32,743Z [root _summarizeReferenceRegion 226] Making summary: 999985 to 1100015 [INFO] 2021-04-23 06:34:32,954Z [root _summarizeReferenceRegion 226] Making summary: 1099985 to 1200015 [INFO] 2021-04-23 06:34:43,680Z [root _fetchChunks 426] Retrieved 584 hits [INFO] 2021-04-23 06:34:47,165Z [root _fetchChunks 426] Retrieved 588 hits [INFO] 2021-04-23 06:34:47,782Z [root _fetchChunks 426] Retrieved 553 hits [INFO] 2021-04-23 06:34:49,053Z [root _fetchChunks 426] Retrieved 666 hits [INFO] 2021-04-23 06:34:50,573Z [root _fetchChunks 426] Retrieved 693 hits [INFO] 2021-04-23 06:34:53,061Z [root _fetchChunks 426] Retrieved 619 hits [INFO] 2021-04-23 06:34:53,171Z [root _fetchChunks 426] Retrieved 709 hits [INFO] 2021-04-23 06:34:53,553Z [root _fetchChunks 426] Retrieved 859 hits [INFO] 2021-04-23 06:34:53,619Z [root _fetchChunks 426] Retrieved 800 hits [INFO] 2021-04-23 06:34:54,199Z [root _fetchChunks 426] Retrieved 1126 hits [INFO] 2021-04-23 06:34:56,197Z [root _fetchChunks 426] Retrieved 940 hits [INFO] 2021-04-23 06:35:02,325Z [root _fetchChunks 426] Retrieved 1154 hits [INFO] 2021-04-23 06:47:30,649Z [root _run 125] Got chunk: (12, ReferenceWindow(refId=0, refName='Chr1', start=1200000, end=1300000)) -- Process: [INFO] 2021-04-23 06:47:33,234Z [root _run 125] Got chunk: (13, ReferenceWindow(refId=0, refName='Chr1', start=1300000, end=1400000)) -- Process: [INFO] 2021-04-23 06:47:33,987Z [root _summarizeReferenceRegion 226] Making summary: 1199985 to 1300015 [INFO] 2021-04-23 06:47:37,258Z [root _summarizeReferenceRegion 226] Making summary: 1299985 to 1400015 [INFO] 2021-04-23 06:47:37,329Z [root _fetchChunks 426] Retrieved 1139 hits [INFO] 2021-04-23 06:47:39,197Z [root _fetchChunks 426] Retrieved 631 hits [INFO] 2021-04-23 06:47:42,357Z [root _run 125] Got chunk: (14, ReferenceWindow(refId=0, refName='Chr1', start=1400000, end=1500000)) -- Process: [INFO] 2021-04-23 06:47:44,612Z [root _run 125] Got chunk: (15, ReferenceWindow(refId=0, refName='Chr1', start=1500000, end=1600000)) -- Process: [INFO] 2021-04-23 06:47:46,085Z [root _run 125] Got chunk: (16, ReferenceWindow(refId=0, refName='Chr1', start=1600000, end=1700000)) -- Process: [INFO] 2021-04-23 06:47:46,402Z [root _summarizeReferenceRegion 226] Making summary: 1399985 to 1500015 [INFO] 2021-04-23 06:47:48,111Z [root _fetchChunks 426] Retrieved 462 hits [INFO] 2021-04-23 06:47:48,588Z [root _summarizeReferenceRegion 226] Making summary: 1499985 to 1600015 [INFO] 2021-04-23 06:47:49,867Z [root _summarizeReferenceRegion 226] Making summary: 1599985 to 1700015 [INFO] 2021-04-23 06:47:51,621Z [root _fetchChunks 426] Retrieved 1129 hits [INFO] 2021-04-23 06:47:51,943Z [root _run 125] Got chunk: (17, ReferenceWindow(refId=0, refName='Chr1', start=1700000, end=1800000)) -- Process: [INFO] 2021-04-23 06:47:54,628Z [root _fetchChunks 426] Retrieved 3289 hits Process KineticWorkerProcess-12: Traceback (most recent call last): File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/usr/local/lib/python3.8/dist-packages/kineticsTools-0.7.0-py3.8-linux-x86_64.egg/kineticsTools/WorkerProcess.py", line 151, in run self._run() File "/usr/local/lib/python3.8/dist-packages/kineticsTools-0.7.0-py3.8-linux-x86_64.egg/kineticsTools/WorkerProcess.py", line 127, in _run result = self.onChunk( File "/usr/local/lib/python3.8/dist-packages/kineticsTools-0.7.0-py3.8-linux-x86_64.egg/kineticsTools/KineticWorker.py", line 121, in onChunk perSiteResults = self._summarizeReferenceRegion( File "/usr/local/lib/python3.8/dist-packages/kineticsTools-0.7.0-py3.8-linux-x86_64.egg/kineticsTools/KineticWorker.py", line 230, in _summarizeReferenceRegion (caseChunks, capValue) = self._fetchChunks( File "/usr/local/lib/python3.8/dist-packages/kineticsTools-0.7.0-py3.8-linux-x86_64.egg/kineticsTools/KineticWorker.py", line 432, in _fetchChunks hits = np.random.choice( File "mtrand.pyx", line 895, in numpy.random.mtrand.RandomState.choice File "/usr/local/lib/python3.8/dist-packages/pbcore-2.3.1-py3.8.egg/pbcore/io/align/BamAlignment.py", line 68, in f raise UnavailableFeature( pbcore.io.align._BamSupport.UnavailableFeature: this feature requires a PacBio BAM index Child process exited with exitcode=1. Aborting. [ERROR] 2021-04-23 06:47:55,550Z [root monitorChildProcesses 592] Child process exited with exitcode=1. Aborting.

Would you help me to fix it? Thank you for your replication.

rhallPB commented 3 years ago

The output suggests an issue with the aligned bam file, the first thing to check would be to generate the pbi index file for the bam https://pbbam.readthedocs.io/en/latest/tools/pbindex.html to see if this solves the issue.