search

PengNi / ccsmeth

Detecting DNA methylation from PacBio CCS reads
BSD 3-Clause Clear License
71 stars 10 forks source link

ccsmeth call_mods #4

Closed SarahChen0401 closed 2 years ago

SarahChen0401 commented 2 years ago

Dear Peng, I run the 'ccsmeth call_mods' command and got these results:

[chenshuhua@xnode07 hg38_hla]$ /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/ccsmeth call_mods \
>   --input ./1_extract_res/${BAM}.pbmm2.features.zscore.fb.depth1.tsv \
>   --model_file /software/ccsmeth/models/model_cpg_attbigru2s_hg002_15kb_s2.b21_epoch7.ckpt \
>   --output ./2_prediction_res/${BAM}.pbmm2.features.zscore.fb.depth1.call_mods.tsv \
>   --threads 16--threads_call 2 --model_type attbigru2s
# ===============================================
## parameters:
input:
    ./1_extract_res/chr22.CKCG-00309-CLR.clr.aln.bam.pbmm2.features.zscore.fb.depth1.tsv
holes_batch:
    50
model_file:
    /software/ccsmeth/models/model_cpg_attbigru2s_hg002_15kb_s2.b21_epoch7.ckpt
model_type:
    attbigru2s
seq_len:
    21
is_stds:
    yes
class_num:
    2
dropout_rate:
    0
batch_size:
    512
n_vocab:
    16
n_embed:
    4
layer_rnn:
    3
hid_rnn:
    256
layer_tfe:
    6
d_model_tfe:
    256
nhead_tfe:
    4
nhid_tfe:
    512
output:
    ./2_prediction_res/chr22.CKCG-00309-CLR.clr.aln.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
ref:
    None
holeids_e:
    None
holeids_ne:
    None
motifs:
    CG
mod_loc:
    0
methy_label:
    1
mapq:
    20
identity:
    0.8
two_strands:
    False
comb_strands:
    False
depth:
    1
norm:
    zscore
no_decode:
    False
num_subreads:
    0
path_to_samtools:
    None
seed:
    1234
threads:
    16
threads_call:
    2
tseed:
    1234
# ===============================================
[main]call_mods starts..
format_features process-767315 starts
format_features process-767309 starts
format_features process-767316 starts
format_features process-767308 starts
read_features process-767307 starts
format_features process-767313 starts
format_features process-767319 starts
format_features process-767312 starts
format_features process-767311 starts
format_features process-767314 starts
format_features process-767318 starts
format_features process-767310 starts
format_features process-767317 starts
write_process-767322 starts
call_mods process-767321 starts
call_mods process-767320 starts
read_features process-767307 ending, read 5292 holes
format_features process-767311 ending, read 9 batches
format_features process-767314 ending, read 7 batches
format_features process-767317 ending, read 9 batches
format_features process-767310 ending, read 6 batches
format_features process-767309 ending, read 10 batches
format_features process-767318 ending, read 7 batches
format_features process-767315 ending, read 9 batches
format_features process-767313 ending, read 9 batches
format_features process-767316 ending, read 8 batches
format_features process-767319 ending, read 11 batches
format_features process-767308 ending, read 11 batches
format_features process-767312 ending, read 10 batches

i can get some results from chr22:

[chenshuhua@login04 2_prediction_res]$ wc -l chr22.CKCG-00309-CLR.clr.aln.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
73499 chr22.CKCG-00309-CLR.clr.aln.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
[chenshuhua@login04 2_prediction_res]$ tail chr22.CKCG-00309-CLR.clr.aln.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
chr22   17985464    +   m64252e_210528_083148/45615225  1   0.048914    0.951086    1   TCCGG
chr22   17985737    +   m64252e_210528_083148/45615225  1   0.64279 0.35721 0   ATCGA
chr22   17981642    +   m64252e_210528_083148/34866157  1   0.052538    0.947462    1   AGCGC
chr22   17981666    +   m64252e_210528_083148/34866157  1   0.065927    0.934073    1   TGCGC
chr22   17981725    +   m64252e_210528_083148/34866157  1   0.12836 0.87164 1   GGCGA
chr22   17981806    +   m64252e_210528_083148/34866157  1   0.174896    0.825104    1   GGCGG
chr22   17982403    +   m64252e_210528_083148/34866157  1   0.056801    0.943199    1   CACGG
chr22   17982495    +   m64252e_210528_083148/34866157  1   0.538114    0.461886    0   AACGC
chr22   17982530    +   m64252e_210528_083148/34866157  1   0.201508    0.798492    1   GGCGC
chr22   17982584    +   m64252e_210528_083148/34866157  1   0.332635    0.667365    1   ATCGC

But it seems like the result just stopped at chr22:17982584 for more than 10 hours.
I check the software status it is "S" meaning "stopping".

top - 09:08:58 up 34 days, 13:44,  2 users,  load average: 1.22, 1.20, 1.23
Tasks: 864 total,   2 running, 860 sleeping,   0 stopped,   2 zombie
%Cpu(s):  2.0 us,  1.3 sy,  0.0 ni, 96.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem : 385166.6 total, 171320.6 free,  20672.6 used, 193173.4 buff/cache
MiB Swap:  51200.0 total,  51097.5 free,    102.5 used. 353409.4 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 693939 chenshu+  20   0   68060   6128   4292 S   0.7   0.0   7:18.51 top
1061537 chenshu+  20   0   65104   5652   3976 R   0.7   0.0   0:00.26 top
 693088 chenshu+  20   0   25172   5952   3400 S   0.0   0.0   0:00.06 /bin/bash
 693850 chenshu+  20   0  172208   6020   4660 S   0.0   0.0   0:02.32 sshd: chenshuhua@pts/2
 693851 chenshu+  20   0   25184   5980   3476 S   0.0   0.0   0:00.05 -bash
 766982 chenshu+  20   0 6323640 119824   6456 S   0.0   0.0   0:04.87 softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/python3 /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/ccsmeth call_mods --input ./1_extract_res/chr2+
 767306 chenshu+  20   0   55196   8924   2464 S   0.0   0.0   0:00.03 /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/python3 -c from multiprocessing.resource_tracker import main;main(12)
 767308 chenshu+  20   0 6802004 519536   6372 S   0.0   0.1   0:07.72 /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/python3 -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=13, pipe_handle=21) --multiprocessing-fork
 767309 chenshu+  20   0 6754884 465660   6352 S   0.0   0.1   0:06.64 /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/python3 -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=13, pipe_handle=23) --multiprocessing-fork
 767310 chenshu+  20   0 6660628 382556   6388 S   0.0   0.1   0:05.01 /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/python3 -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=13, pipe_handle=26) --multiprocessing-fork
 767311 chenshu+  20   0 6702432 424536   6492 S   0.0   0.1   0:06.56 /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/python3 -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=13, pipe_handle=29) --multiprocessing-fork
 767312 chenshu+  20   0 6720200 437152   6480 S   0.0   0.1   0:06.44 /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/python3 -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=13, pipe_handle=31) --multiprocessing-fork
 767313 chenshu+  20   0 6710220 377140   6340 S   0.0   0.1   0:06.41 /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/python3 -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=13, pipe_handle=34) --multiprocessing-fork
 767314 chenshu+  20   0 6747560 423872   6440 S   0.0   0.1   0:05.63 /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/python3 -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=13, pipe_handle=37) --multiprocessing-fork
 767315 chenshu+  20   0 6706640 372192   6408 S   0.0   0.1   0:06.33 /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/python3 -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=13, pipe_handle=39) --multiprocessing-fork
 767316 chenshu+  20   0 6717492 436604   6436 S   0.0   0.1   0:06.16 /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/python3 -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=13, pipe_handle=41) --multiprocessing-fork
 767317 chenshu+  20   0 6743736 414116   6368 S   0.0   0.1   0:06.23 /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/python3 -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=13, pipe_handle=43) --multiprocessing-fork
 767318 chenshu+  20   0 6640932 360924   6364 S   0.0   0.1   0:05.28 /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/python3 -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=13, pipe_handle=45) --multiprocessing-fork
 767319 chenshu+  20   0 6770504 483744   6368 S   0.0   0.1   0:06.85 /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/python3 -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=13, pipe_handle=47) --multiprocessing-fork
 767320 chenshu+  20   0       0      0      0 Z   0.0   0.0   0:08.16 [python3] <defunct>
 767321 chenshu+  20   0       0      0      0 Z   0.0   0.0   0:08.17 [python3] <defunct>
 767322 chenshu+  20   0 6323268 123896   6640 S   0.0   0.0   0:06.85 /softwares/ccsmeth_v0.1.1_d20220329_python3venv/bin/python3 -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=13, pipe_handle=53) --multiprocessing-fork
1061449 chenshu+  20   0  172208   5636   4272 S   0.0   0.0   0:00.00 sshd: chenshuhua@pts/3
1061450 chenshu+  20   0   25184   5904   3400 S   0.0   0.0   0:00.06 -bash

I was wondering if the problem is from my data, my GPU, or your software?

PengNi commented 2 years ago

Hi @SarahChen0401 , I'm not sure what was stucked there. Maybe it is because of the code of ccsmeth, maybe it is because of the other processes running in this machine. I suggest you re-submit this job and see what happens.

Also, how many lines does your input (./1_extract_res/chr22.CKCG-00309-CLR.clr.aln.bam.pbmm2.features.zscore.fb.depth1.tsv) have? Do you have a GPU in this machine? If you do, can you show me the nvidia-smi output?

Best, Peng

SarahChen0401 commented 2 years ago

Hi Peng, there are 15620865 lines in my input (./1_extract_res/chr22.CKCG-00309-CLR.clr.aln.bam.pbmm2.features.zscore.fb.depth1.tsv). 

[chenshuhua@login04 1_extract_res]$ wc -l chr22.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.tsv
15620865 chr22.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.tsv

I do have a GPU on my machine. Here is the nvidia-smi output:

[chenshuhua@gvno02 ~]$ nvidia-smi
Mon Apr 18 13:38:54 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla V100-SXM2...  Off  | 00000000:1B:00.0 Off |                    0 |
| N/A   29C    P0    52W / 300W |   3741MiB / 32510MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  Tesla V100-SXM2...  Off  | 00000000:1C:00.0 Off |                    0 |
| N/A   26C    P0    52W / 300W |   1874MiB / 32510MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   2  Tesla V100-SXM2...  Off  | 00000000:60:00.0 Off |                    0 |
| N/A   28C    P0    54W / 300W |   3745MiB / 32510MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   3  Tesla V100-SXM2...  Off  | 00000000:62:00.0 Off |                    0 |
| N/A   27C    P0    51W / 300W |   3741MiB / 32510MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   4  Tesla V100-SXM2...  Off  | 00000000:B1:00.0 Off |                    0 |
| N/A   25C    P0    51W / 300W |   3741MiB / 32510MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   5  Tesla V100-SXM2...  Off  | 00000000:B2:00.0 Off |                    0 |
| N/A   28C    P0    51W / 300W |   3741MiB / 32510MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   6  Tesla V100-SXM2...  Off  | 00000000:DA:00.0 Off |                    0 |
| N/A   24C    P0    52W / 300W |   3745MiB / 32510MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   7  Tesla V100-SXM2...  Off  | 00000000:DC:00.0 Off |                    0 |
| N/A   26C    P0    53W / 300W |      0MiB / 32510MiB |      2%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A   3878703      C   ...9_python3venv/bin/python3     1869MiB |
|    0   N/A  N/A   3878706      C   ...9_python3venv/bin/python3     1869MiB |
|    1   N/A  N/A   3878711      C   ...9_python3venv/bin/python3     1871MiB |
|    2   N/A  N/A   3275195      C   ...9_python3venv/bin/python3     1869MiB |
|    2   N/A  N/A   3275198      C   ...9_python3venv/bin/python3     1871MiB |
|    3   N/A  N/A   3166020      C   ...9_python3venv/bin/python3     1869MiB |
|    3   N/A  N/A   3166021      C   ...9_python3venv/bin/python3     1869MiB |
|    4   N/A  N/A   4119752      C   ...9_python3venv/bin/python3     1869MiB |
|    4   N/A  N/A   4119753      C   ...9_python3venv/bin/python3     1869MiB |
|    5   N/A  N/A   3275201      C   ...9_python3venv/bin/python3     1869MiB |
|    5   N/A  N/A   3275202      C   ...9_python3venv/bin/python3     1869MiB |
|    6   N/A  N/A   3878704      C   ...9_python3venv/bin/python3     1871MiB |
|    6   N/A  N/A   3878707      C   ...9_python3venv/bin/python3     1869MiB |
+-----------------------------------------------------------------------------+

As you mentioned, I submitted 22 chromosomes + X,Y,M chromosomes to run ccsmeth call_mods on several gpu nodes. they were running more than 6 days, 

[chenshuhua@login04 ~]$ squeue | grep chenshu | grep ccsmeth3
       13566908_19 rt-2080ti ccsmeth3 chenshuh  R 6-06:23:16      1 grtq16
       13566908_20 rt-2080ti ccsmeth3 chenshuh  R 6-06:23:16      1 grtq16
       13566908_21 rt-2080ti ccsmeth3 chenshuh  R 6-06:23:16      1 grtq16
       13566908_23 rt-2080ti ccsmeth3 chenshuh  R 6-06:23:16      1 grtq16
       13566908_17 rt-2080ti ccsmeth3 chenshuh  R 8-21:36:56      1 grtq14
       13566908_18 rt-2080ti ccsmeth3 chenshuh  R 8-21:36:56      1 grtq14
        13566893_1 rt-2080ti ccsmeth3 chenshuh  R 10-02:59:41      1 grtq14
        13566893_2 rt-2080ti ccsmeth3 chenshuh  R 10-02:59:41      1 grtq14
        13566893_3 rt-2080ti ccsmeth3 chenshuh  R 10-02:59:41      1 grtq15
        13566893_4 rt-2080ti ccsmeth3 chenshuh  R 10-02:59:41      1 grtq15
        13566893_5 rt-2080ti ccsmeth3 chenshuh  R 10-02:59:41      1 grtq15
       13566908_16  v100-oct ccsmeth3 chenshuh  R 8-22:22:33      1 gvno02
       13566908_13  v100-oct ccsmeth3 chenshuh  R 9-06:20:59      1 gvno02
       13566908_14  v100-oct ccsmeth3 chenshuh  R 9-06:20:59      1 gvno02
       13566908_15  v100-oct ccsmeth3 chenshuh  R 9-06:20:59      1 gvno02
        13566908_9  v100-oct ccsmeth3 chenshuh  R 9-11:48:30      1 gvno01
       13566908_10  v100-oct ccsmeth3 chenshuh  R 9-11:48:30      1 gvno01
       13566908_11  v100-oct ccsmeth3 chenshuh  R 9-11:48:30      1 gvno01
       13566908_12  v100-oct ccsmeth3 chenshuh  R 9-11:48:30      1 gvno01
        13566908_7  v100-oct ccsmeth3 chenshuh  R 9-23:38:22      1 gvno02
        13566908_8  v100-oct ccsmeth3 chenshuh  R 9-23:38:22      1 gvno02
        13566893_6  v100-oct ccsmeth3 chenshuh  R 10-02:59:31      1 gvno02

these showed time that results updated last time:

[chenshuhua@login04 2_prediction_res]$ ls -lhrt
total 5.7G
-rw-r--r-- 1 chenshuhua yangjian  51M Apr  8 10:27 chr3.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian 112M Apr  8 10:29 chr5.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian 291M Apr  8 10:30 chr6.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian 364M Apr  8 10:32 chr1.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian 425M Apr  8 10:33 chr4.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian 352M Apr  8 10:36 chr2.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian 1.2G Apr  8 11:48 1_chr22.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian 350M Apr  8 13:52 chr7.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian 351M Apr  8 13:53 chr8.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian  25M Apr  9 01:38 chr12.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian  32M Apr  9 01:39 chr10.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian  30M Apr  9 01:39 chr11.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian  70M Apr  9 01:41 chr9.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian 286M Apr  9 07:10 chr15.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian 436M Apr  9 07:11 chr13.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian 432M Apr  9 07:14 chr14.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian 332M Apr  9 15:09 chr16.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian  45M Apr  9 15:51 chr18.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian 390M Apr  9 15:57 chr17.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian  48M Apr 12 07:04 chr19.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian  43M Apr 12 07:04 chr20.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian  45M Apr 12 07:05 chr21.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian  47M Apr 12 07:05 chrX.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian 126M Apr 13 15:32 chrY.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv
-rw-r--r-- 1 chenshuhua yangjian 1.6M Apr 13 15:33 chrM.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv

it seems the result stuck at different positions in different chromosome:

[chenshuhua@login04 2_prediction_res]$ tail -n1  *
==> 1_chr22.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr22   50424885    +   m64200e_210430_130607/56165855  7   0.132108    0.867892    1   CCCGC

==> chr10.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr10   799548  +   m64189e_210427_070436/22413697  3   0.358548    0.641452    1   CACGC

==> chr11.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr11   1374051 +   m64189e_210427_070436/48497010  1   0.035052    0.964948    1   TCCGG

==> chr12.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr12   852764  +   m64189e_210427_070436/135596881 5   0.677823    0.322177    0   AACGA

==> chr13.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr13   39428504    +   m64189e_210427_070436/30411209  3   0.141472    0.858528    1   CCCGT

==> chr14.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr14   43990296    +   m64189e_210427_070436/123994691 5   0.682539    0.317461    0   AACGA

==> chr15.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr15   40283915    +   m64200e_210430_130607/20578596  2   0.146907    0.853093    1   GCCGC

==> chr16.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr16   8718054 +   m64189e_210427_070436/139068357 3   0.029902    0.970098    1   GCCGT

==> chr17.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr17   16570842    +   m64189e_210427_070436/55575669  1   0.893883    0.106117    0   CACGA

==> chr18.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr18   2788336 +   m64189e_210427_070436/169806003 3   0.219353    0.780647    1   ATCGT

==> chr19.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr19   1469032 +   m64200e_210430_130607/28967102  6   0.997378    0.002622    0   AACGC

==> chr1.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr1    10391021    +   m64189e_210427_070436/168297676 5   0.56864 0.43136 0   CACGT

==> chr20.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr20   838557  +   m64200e_210430_130607/22611007  5   0.400747    0.599253    1   CTCGG

==> chr21.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr21   9033780 +   m64200e_210430_130607/50398251  5   0.31292 0.68708 1   CCCGG

==> chr2.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr2    15008617    +   m64189e_210427_070436/41617546  7   0.004072    0.995928    1   ATCGG

==> chr3.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr3    3194675 +   m64200e_210430_130607/59704157  2   0.664408    0.335592    0   AACGC

==> chr4.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr4    14500250    +   m64189e_210427_070436/141164661 7   0.000319    0.999681    1   ATCGA

==> chr5.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr5    4495723 +   m64189e_210427_070436/114164665 3   0.926066    0.073934    0   CTCGT

==> chr6.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr6    12452371    +   m64189e_210427_070436/57606513  1   0.190843    0.809157    1   AGCGA

==> chr7.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr7    11049452    +   m64200e_210430_130607/88408785  1   0.354293    0.645707    1   GGCGA

==> chr8.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr8    17797788    +   m64189e_210427_070436/128779112 3   0.017365    0.982635    1   TACGC

==> chr9.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chr9    4107563 +   m64200e_210430_130607/111149217 2   0.08882 0.91118 1   GCCGC

==> chrM.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chrM    16541   +   m64189e_210427_070436/91620274  4   0.511339    0.488661    0   CACGT

==> chrX.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chrX    1673712 +   m64200e_210430_130607/4589605   3   0.406132    0.593868    1   CCCGG

==> chrY.CKCG-00309-HiFi.hifi.subreads.bam.pbmm2.features.zscore.fb.depth1.call_mods.tsv <==
chrY    56886943    +   m64189e_210427_070436/38798596  5   0.657119    0.342881    0   CACGT

Oh, I run over chr22 for one time, but I can't not run over it again after that time, and also can not run through on other chromosomes.

Best wishes, Shuhua

PengNi commented 2 years ago

@SarahChen0401 , thanks for clarifying these. Everything seems fine and ccsmeth should work in your case. One possible reason is that you submitted too many jobs at the same time, but this machine doesn't have enough resources so every job was stucked.

It seems you applied 16 processors/threads for each job. So How many processors does your machine have? And how many jobs did you submit at the same time?

Best, Peng

SarahChen0401 commented 2 years ago

@SarahChen0401 , thanks for clarifying these. Everything seems fine and ccsmeth should work in your case. One possible reason is that you submitted too many jobs at the same time, but this machine doesn't have enough resources so every job was stucked.

It seems you applied 16 processors/threads for each job. So How many processors does your machine have? And how many jobs did you submit at the same time?

Best, Peng

Thanks Peng. I applied 2 threads for each job, it works!

Best wishes, Shuhua