Closed nbargues closed 3 months ago
Hi Nicolas,
Could you please provide $Ncpu as well? I can probably reproduce the error easier with it and it helps me to debug.
Also, now I noticed that you plan to identify a frameshift instead of copy number change. I was under the impression that you want to identify number of repeats (as most people do this) and suggested --pacbio
would be useful.
Our frameshift identification won't perform very well with error prone reads and we use it only with short reads. I would suggest to use a tool specifically designed for this task with nanopore reads and extensively tested for it. There are multiple such tools and I am aware of longshot which I recommend you to use instead of our tool.
Thank you for reporting the issue with multiprocessing.
Hi, I sequence long read of the gene MUC1 of a patient that I know is positif to the MUC1 mutation causal of a disease ( that is in your database ) and I want , via your software, confirm that this mutation is present. Do you think that your software is capable of that ? and what are the best parameters for that ?
In the previous example, $Ncpu = 18.
Edit : I re-try the command but without --frameshift and -t argument and the same error occur
Yes. We can do it for MUC1 with accurate short reads but for long error prone reads our approach is not the best (shorter reads are not disadvantage for us like GATK, etc, but high error rate is a problem for frameshift identification). If I remember correctly, this VNTR is ~1000bp so using nanopore reads this case shouldn't be different with any other SNV for long read variant callers like longshot.
Thank you for providing the information about the bug. We work to resolve it independently.
Hi, apparently I have a multiprocess error when running adVNTR ; the command :
advntr genotype --alignment_file $bam --working_directory $advntrDir/$bam_name --vntr_id 25561 --pacbio --frameshift -m ../hg19_selected_VNTRs_Pacbio.db -t $Ncpu
the error :
[M::bam2fq_mainloop] discarded 0 singletons [M::bam2fq_mainloop] processed 14991 reads Process Process-1009: Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 749, in _callmethod conn = self._tls.connection AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 264, in check_if_pacbio_read_spans_vntr self.check_if_flanking_regions_align_to_str(str(read.seq).upper(), length_distribution, spanning_reads) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 260, in check_if_flanking_regions_align_to_str spanning_reads.append(read_str[left_align[3]:right_align[3]+flanking_region_size]) File "", line 2, in append
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 753, in _callmethod
self._connect()
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 740, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 487, in Client
c = SocketClient(address)
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient
s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
Process Process-1017:
Traceback (most recent call last):
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 749, in _callmethod
conn = self._tls.connection
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 264, in check_if_pacbio_read_spans_vntr self.check_if_flanking_regions_align_to_str(str(read.seq).upper(), length_distribution, spanning_reads) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 260, in check_if_flanking_regions_align_to_str spanning_reads.append(read_str[left_align[3]:right_align[3]+flanking_region_size]) File "", line 2, in append
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 753, in _callmethod
self._connect()
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 740, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 487, in Client
c = SocketClient(address)
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient
s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
Process Process-1010:
Traceback (most recent call last):
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 749, in _callmethod
conn = self._tls.connection
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 264, in check_if_pacbio_read_spans_vntr self.check_if_flanking_regions_align_to_str(str(read.seq).upper(), length_distribution, spanning_reads) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 260, in check_if_flanking_regions_align_to_str spanning_reads.append(read_str[left_align[3]:right_align[3]+flanking_region_size]) File "", line 2, in append
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 753, in _callmethod
self._connect()
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 740, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 487, in Client
c = SocketClient(address)
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient
s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
Process Process-994:
Traceback (most recent call last):
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 749, in _callmethod
conn = self._tls.connection
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 264, in check_if_pacbio_read_spans_vntr self.check_if_flanking_regions_align_to_str(str(read.seq).upper(), length_distribution, spanning_reads) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 260, in check_if_flanking_regions_align_to_str spanning_reads.append(read_str[left_align[3]:right_align[3]+flanking_region_size]) File "", line 2, in append
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 753, in _callmethod
self._connect()
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 740, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 487, in Client
c = SocketClient(address)
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient
s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
Process Process-1003:
Traceback (most recent call last):
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 749, in _callmethod
conn = self._tls.connection
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, self._kwargs) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 264, in check_if_pacbio_read_spans_vntr self.check_if_flanking_regions_align_to_str(str(read.seq).upper(), length_distribution, spanning_reads) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 260, in check_if_flanking_regions_align_to_str spanning_reads.append(read_str[left_align[3]:right_align[3]+flanking_region_size]) File "", line 2, in append
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 753, in _callmethod
self._connect()
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 740, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 487, in Client
c = SocketClient(address)
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient
s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
Traceback (most recent call last):
File "/opt/miniconda3/envs/gene36/bin/advntr", line 11, in
sys.exit(main())
File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/main.py", line 121, in main
genotype(args, genotype_parser)
File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/advntr_commands.py", line 101, in genotype
genome_analyzier.find_repeat_counts_from_pacbio_alignment_file(input_file)
File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/genome_analyzer.py", line 103, in find_repeat_counts_from_pacbio_alignment_file
copy_numbers = self.vntr_finder[vid].find_repeat_count_from_pacbio_alignment_file(alignment_file, reads)
File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/profiler.py", line 8, in wrapper
retval = func(*args, *kwargs)
File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 477, in find_repeat_count_from_pacbio_alignment_file
mapped_spanning_reads = self.get_spanning_reads_of_aligned_pacbio_reads(alignment_file)
File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/profiler.py", line 8, in wrapper
retval = func(args, kwargs)
File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 329, in get_spanning_reads_of_aligned_pacbio_reads
p.start()
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init
self._launch(process_obj)
File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/popen_fork.py", line 65, in _launch
parent_r, child_w = os.pipe()
OSError: [Errno 24] Too many open files