search

czbiohub-sf / orpheum

Orpheum (Previously called and published under sencha) is a Python package for directly translating RNA-seq reads into coding protein sequence.
MIT License
18 stars 4 forks source link

Revert "multiprocessing for sencha translate" #91

Closed pranathivemuri closed 4 years ago

pranathivemuri commented 4 years ago

Reverts czbiohub/sencha#89

having bunch of errors while running sencha translate as part of kmermaid pipelines on different data - so reverting it

pranathivemuri commented 4 years ago 
      result = (True, func(*args, **kwds))
    File "/home/pranathi/miniconda3/envs/nf-core-kmermaid-1.0.0dev/lib/python3.6/site-packages/multiprocess/pool.py", line 44, in mapstar
      return list(map(*args))
    File "/home/pranathi/.local/lib/python3.6/site-packages/sencha-1.0.1.dev78+g73401f4-py3.6.egg/sencha/translate.py", line 333, in maybe_score_single_read
      if evaluate_is_fastp_low_complexity(sequence):
    File "/home/pranathi/.local/lib/python3.6/site-packages/sencha-1.0.1.dev78+g73401f4-py3.6.egg/sencha/translate.py", line 78, in evaluate_is_fastp_low_complexity
      complexity = compute_fastp_complexity(seq)
    File "/home/pranathi/.local/lib/python3.6/site-packages/sencha-1.0.1.dev78+g73401f4-py3.6.egg/sencha/translate.py", line 86, in compute_fastp_complexity
      complexity = n_different_consecutively / len(seq)
  ZeroDivisionError: division by zero
  """

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "/home/pranathi/miniconda3/envs/nf-core-kmermaid-1.0.0dev/bin/sencha", line 8, in <module>
      sys.exit(cli())
    File "/home/pranathi/miniconda3/envs/nf-core-kmermaid-1.0.0dev/lib/python3.6/site-packages/click/core.py", line 829, in __call__
      return self.main(*args, **kwargs)
    File "/home/pranathi/miniconda3/envs/nf-core-kmermaid-1.0.0dev/lib/python3.6/site-packages/click/core.py", line 782, in main
      rv = self.invoke(ctx)
    File "/home/pranathi/miniconda3/envs/nf-core-kmermaid-1.0.0dev/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/home/pranathi/miniconda3/envs/nf-core-kmermaid-1.0.0dev/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/home/pranathi/miniconda3/envs/nf-core-kmermaid-1.0.0dev/lib/python3.6/site-packages/click/core.py", line 610, in invoke
      return callback(*args, **kwargs)
    File "/home/pranathi/.local/lib/python3.6/site-packages/sencha-1.0.1.dev78+g73401f4-py3.6.egg/sencha/translate.py", line 683, in cli
      translate_obj.set_coding_scores_all_files()
    File "/home/pranathi/.local/lib/python3.6/site-packages/sencha-1.0.1.dev78+g73401f4-py3.6.egg/sencha/translate.py", line 458, in set_coding_scores_all_files
      self.score_reads_per_file(reads_file)
    File "/home/pranathi/.local/lib/python3.6/site-packages/sencha-1.0.1.dev78+g73401f4-py3.6.egg/sencha/translate.py", line 401, in score_reads_per_file
      pool.map(self.maybe_score_single_read, fasta_files_split)
    File "/home/pranathi/miniconda3/envs/nf-core-kmermaid-1.0.0dev/lib/python3.6/site-packages/multiprocess/pool.py", line 266, in map
      return self._map_async(func, iterable, mapstar, chunksize).get()
    File "/home/pranathi/miniconda3/envs/nf-core-kmermaid-1.0.0dev/lib/python3.6/site-packages/multiprocess/pool.py", line 644, in get
      raise self._value
  ZeroDivisionError: division by zero

Work dir:
  /home/pranathi/code/nf-core/kmermaid/work/b0/bf40a095328aac2aa3d6a431ab1a12

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

(nf-core-kmermaid-1.0.0dev) 1 pranathi@ndnd:~/code/nf-core/kmermaid$ cd /home/pranathi/code/nf-core/kmermaid/work/b0/bf40a095328aac2aa3d6a431ab1a12
(nf-core-kmermaid-1.0.0dev) pranathi@ndnd:~/code/nf-core/kmermaid/work/b0/bf40a095328aac2aa3d6a431ab1a12$ ls
TSP2_Thymus_NA_1_1_possorted_genome_bam__unaligned__CGAGTTATCTGTGTGA__coding_reads_nucleotides.fasta
TSP2_Thymus_NA_1_1_possorted_genome_bam__unaligned__CGAGTTATCTGTGTGA__coding_reads_peptides.fasta
TSP2_Thymus_NA_1_1_possorted_genome_bam__unaligned__CGAGTTATCTGTGTGA__noncoding_reads_nucleotides.fasta
TSP2_Thymus_NA_1_1_possorted_genome_bam__unaligned__CGAGTTATCTGTGTGA_R1_trimmed.fastq.gz
uniprot-reviewed_yes+taxonomy_Mammalia_9MAMM_40674__molecule-protein_ksize-9.bloomfilter
(nf-core-kmermaid-1.0.0dev) pranathi@ndnd:~/code/nf-core/kmermaid/work/b0/bf40a095328aac2aa3d6a431ab1a12$ ls -al
total 688
drwxrwxr-x     2 pranathi pranathi   4096 Aug 27 02:33 .
drwxrwxr-x 11600 pranathi pranathi 659456 Aug 27 02:32 ..
-rw-rw-r--     1 pranathi pranathi      0 Aug 27 02:33 .command.begin
-rw-rw-r--     1 pranathi pranathi   3194 Aug 27 02:33 .command.err
-rw-rw-r--     1 pranathi pranathi   3194 Aug 27 02:33 .command.log
-rw-rw-r--     1 pranathi pranathi      0 Aug 27 02:33 .command.out
-rw-rw-r--     1 pranathi pranathi   9920 Aug 27 02:33 .command.run
-rw-rw-r--     1 pranathi pranathi    910 Aug 27 02:33 .command.sh
-rw-rw-r--     1 pranathi pranathi      0 Aug 27 02:33 .command.trace
-rw-rw-r--     1 pranathi pranathi      1 Aug 27 02:33 .exitcode
-rw-rw-r--     1 pranathi pranathi      0 Aug 27 02:33 TSP2_Thymus_NA_1_1_possorted_genome_bam__unaligned__CGAGTTATCTGTGTGA__coding_reads_nucleotides.fasta
-rw-rw-r--     1 pranathi pranathi      0 Aug 27 02:33 TSP2_Thymus_NA_1_1_possorted_genome_bam__unaligned__CGAGTTATCTGTGTGA__coding_reads_peptides.fasta
-rw-rw-r--     1 pranathi pranathi      0 Aug 27 02:33 TSP2_Thymus_NA_1_1_possorted_genome_bam__unaligned__CGAGTTATCTGTGTGA__noncoding_reads_nucleotides.fasta
lrwxrwxrwx     1 pranathi pranathi    164 Aug 27 02:33 TSP2_Thymus_NA_1_1_possorted_genome_bam__unaligned__CGAGTTATCTGTGTGA_R1_trimmed.fastq.gz -> /home/pranathi/code/nf-core/kmermaid/work/64/ab31d0b6f4e352286cb19d80da3a65/TSP2_Thymus_NA_1_1_possorted_genome_bam__unaligned__CGAGTTATCTGTGTGA_R1_trimmed.fastq.gz

sencha translate \
  --processes 10 \
  --intermediate-directory /mnt/ibm_lg/pranathi/immune-evolution/sencha_intermediate2/ \
  --molecule dayhoff \
  --coding-nucleotide-fasta TSP2_Lung_proxmedialdistal_1_1_possorted_genome_bam__unaligned__GGGTATTTCAAGAGGC__coding_reads_nucleotides.fasta \
  --noncoding-nucleotide-fasta TSP2_Lung_proxmedialdistal_1_1_possorted_genome_bam__unaligned__GGGTATTTCAAGAGGC__noncoding_reads_nucleotides.fasta \
  --csv TSP2_Lung_proxmedialdistal_1_1_possorted_genome_bam__unaligned__GGGTATTTCAAGAGGC__coding_scores.csv \
  --json-summary TSP2_Lung_proxmedialdistal_1_1_possorted_genome_bam__unaligned__GGGTATTTCAAGAGGC__coding_summary.json \
  --jaccard-threshold 0.95 \
  --peptide-ksize 9 \
  --peptides-are-bloom-filter \
  metazoa_odb10__molecule-dayhoff_ksize-9.bloomfilter \
  TSP2_Lung_proxmedialdistal_1_1_possorted_genome_bam__unaligned__GGGTATTTCAAGAGGC_R1_trimmed.fastq.gz > TSP2_Lung_proxmedialdistal_1_1_possorted_genome_bam__unaligned__GGGTATTTCAAGAGGC__coding_reads_peptides.fasta

  11it [00:00, 7771.15it/s]
8it [00:00, 12524.98it/s]
6it [00:00, 7155.48it/s]
5it [00:00, 7833.96it/s]
8it [00:00, 13454.06it/s]
10it [00:00, 5638.26it/s]
15it [00:00, 25890.77it/s]
8it [00:00, 8481.91it/s]
6it [00:00, 15363.75it/s]
1it [00:00, 3368.92it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
3it [00:00, 403.84it/s]
3it [00:00, 794.48it/s]
2it [00:00, 2177.16it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
Traceback (most recent call last):
  File "/home/pranathi/miniconda3/envs/nf-core-kmermaid-1.0.0dev/bin/sencha", line 8, in <module>
    sys.exit(cli())
  File "/home/pranathi/miniconda3/envs/nf-core-kmermaid-1.0.0dev/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/pranathi/miniconda3/envs/nf-core-kmermaid-1.0.0dev/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/pranathi/miniconda3/envs/nf-core-kmermaid-1.0.0dev/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/pranathi/miniconda3/envs/nf-core-kmermaid-1.0.0dev/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/pranathi/miniconda3/envs/nf-core-kmermaid-1.0.0dev/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/pranathi/.local/lib/python3.6/site-packages/sencha-1.0.1.dev65+g0e1d781-py3.6.egg/sencha/translate.py", line 672, in cli
    translate_obj.set_coding_scores_all_files()
  File "/home/pranathi/.local/lib/python3.6/site-packages/sencha-1.0.1.dev65+g0e1d781-py3.6.egg/sencha/translate.py", line 450, in set_coding_scores_all_files
    self.score_reads_per_file(reads_file)
  File "/home/pranathi/.local/lib/python3.6/site-packages/sencha-1.0.1.dev65+g0e1d781-py3.6.egg/sencha/translate.py", line 440, in score_reads_per_file
    float(row[2]),
ValueError: could not convert string to float: 'TSP2_Lung_proxmedialdistal_1_1_possorted_genome_bam__unaligned__GGGTATTTCAAGAGGC_R1_trimmed.fastq.gz'
(nf-c

another error
row indexing error