atropos qc multithreads not working ?

aathbt commented 4 years ago

Hello,

A few days ago, I tried to use the multithreads option in atropos qc to deal with large fastq files, and it kept crashing.

These are the commands I tried :

atropos qc -pe1 test_1.fastq -pe2 test_2.fastq -T 8
atropos qc -pe1 test_1.fastq -pe2 test_2.fastq -T 8 --process-timeout 120
atropos qc -se SRR5345750.fastq -T 8
atropos qc -se SRR5345750.fastq -T 8 --process-timeout 120

Everytime, I obtained the same error log :

2020-05-29 19:48:48,124 INFO: This is Atropos 1.1.26+1.g9e38452 with Python 3.6.9
2020-05-29 19:48:48,125 ERROR: Error executing command qc  
Traceback (most recent call last):
  File "/home/antonin/.local/lib/python3.6/site-packages/atropos-1.1.26+1.g9e38452-py3.6-linux-x86_64.egg/atropos/commands/base.py", line 332, in run
    self.return_code = self()
  File "/home/antonin/.local/lib/python3.6/site-packages/atropos-1.1.26+1.g9e38452-py3.6-linux-x86_64.egg/atropos/commands/qc/__init__.py", line 66, in __call__
    return self.run_parallel(pipeline_class, pipeline_args)
  File "/home/antonin/.local/lib/python3.6/site-packages/atropos-1.1.26+1.g9e38452-py3.6-linux-x86_64.egg/atropos/commands/qc/__init__.py", line 83, in run_parallel
    self.threads, self.timeout)
  File "/home/antonin/.local/lib/python3.6/site-packages/atropos-1.1.26+1.g9e38452-py3.6-linux-x86_64.egg/atropos/commands/base.py", line 241, in __getattr__
    raise ValueError("Unknown attribute: {}".format(name))
ValueError: Unknown attribute: timeout

Notes :

I reproduced this error with smaller files.
These files are perfectly processed by atropos trim using multithreads.

Hope you can find what's causing that ! Many thanks !

jdidion commented 4 years ago

Thanks for reporting. I've fixed this and released a new version (1.1.27).

aathbt commented 4 years ago

Hello @jdidion

I tried version 1.1.27 with the same datasets and commands than before, and I got a new error :

2020-06-02 15:23:26,131 INFO: This is Atropos 1.1.27 with Python 3.6.9
2020-06-02 15:23:26,132 ERROR: Error executing command qc
Traceback (most recent call last):
  File "/home/antonin/.local/lib/python3.6/site-packages/atropos-1.1.27-py3.6-linux-x86_64.egg/atropos/commands/base.py", line 332, in run
    self.return_code = self()
  File "/home/antonin/.local/lib/python3.6/site-packages/atropos-1.1.27-py3.6-linux-x86_64.egg/atropos/commands/qc/__init__.py", line 66, in __call__
    return self.run_parallel(pipeline_class, pipeline_args)
  File "/home/antonin/.local/lib/python3.6/site-packages/atropos-1.1.27-py3.6-linux-x86_64.egg/atropos/commands/qc/__init__.py", line 82, in run_parallel
    'QcPipelineImpl', (ParallelPipelineMixin, pipeline_class))
TypeError: type() takes 1 or 3 arguments

Thanks in advance for you help !

jdidion commented 4 years ago

Thanks - working on this now

jdidion commented 4 years ago

Sorry about that - I rushed out the previous fix without proper testing. It is now working in 1.1.28.

aathbt commented 4 years ago

No worries. It works perfectly now !

jdidion / atropos

atropos qc multithreads not working ? #100