search

merenlab / anvio

An analysis and visualization platform for 'omics data
http://merenlab.org/software/anvio
GNU General Public License v3.0
413 stars 142 forks source link

Anvio profiling assistance #2264

Closed kinosham closed 1 month ago

kinosham commented 1 month ago

Good day.

I am very new to bioinformatics. Please can you help. I am trying to run this command :

!/bin/bash

PBS -l nodes=1:ppn=24

PBS -l walltime=720:00:00

PBS -q bigmem

PBS -o /nlustre/users/kinosha/Hammanskraal_profiling_K1_kinosha_output

PBS -e /nlustre/users/kinosha/Hammanskraal_profiling_K1_kinosha_error

PBS -k oe

PBS -m bae

PBS -M kinoshamoodley@gmail.com

cd /nlustre/users/kinosha

module load anvio8

source /apps/anaconda3-2023.03/etc/profile.d/conda.sh conda activate anvio-8

anvi-init-bam /nlustre/users/kinosha/K1.bam -o /nlustre/users/kinosha/Hammanskraal_metegonomics_sequences/Binning/K1.sorted.bam anvi-profile -i/nlustre/users/kinosha/Hammanskraal_metegonomics_sequences/Binning/K1.sorted.bam -c /nlustre/users/kinosha/Hammanskraal_metegonomics_sequences/Binning/contigs.db -T 16 --skip-SNV-profiling -M 1000

But keep getting this error: You should now please run:

source /apps/anaconda3-2023.03/etc/profile.d/conda.sh conda activate anvio-8 Sorted BAM File ..............................: /nlustre/users/kinosha/Hammanskraal_metegonomics_sequences/Binning/K2.sorted.bam BAM File Index ...............................: /nlustre/users/kinosha/Hammanskraal_metegonomics_sequences/Binning/K2.sorted.bam.bai Sample name set ...................................: K2_sorted Description .......................................: None Profile DB path ...................................: /nlustre/users/kinosha/Hammanskraal_metegonomics_sequences/Binning/K2_sorted/PROFILE.db Contigs DB path ...................................: /nlustre/users/kinosha/Hammanskraal_metegonomics_sequences/contigs.db Contigs DB hash ...................................: hash84508bed Command line ......................................: /apps/anaconda3-2023.03/envs/anvio-8/bin/anvi-profile -i/nlustre/users/kinosha/Hammanskraal_metegonomics_sequences/Binning/K2.sorted.bam -c /nlustre/users/kinosha/Hammanskraal_metegonomics_sequences/contigs.db -T 16 --skip-SNV-profiling -M 1000

Minimum percent identity of reads to be profiled ..: None Fetch filter engaged ..............................: None

Is merged profile? ................................: False Is blank profile? .................................: False Skip contigs shorter than .........................: 1,000 Skip contigs longer than ..........................: 9,223,372,036,854,775,807 Perform hierarchical clustering of contigs? .......: False

Profile single-nucleotide variants (SNVs)? ........: False Profile single-codon variants (SCVs/+SAAVs)? ......: False Profile insertion/deletions (INDELs)? .............: False Minimum coverage to calculate SNVs ................: 10 Report FULL variability data? .....................: False

WARNING

Your minimum contig length is set to 1,000 base pairs. So anvi'o will not take into consideration anything below that. If you need to kill this an restart your analysis with another minimum contig length value, feel free to press CTRL+C.

Input BAM .........................................: /nlustre/users/kinosha/Hammanskraal_metegonomics_sequences/Binning/K2.sorted.bam Output directory path .............................: /nlustre/users/kinosha/Hammanskraal_metegonomics_sequences/Binning/K2_sorted

Number of reads in the BAM file ...................: 187,230,548 Number of sequences in the contigs DB .............: 1,990,377 Number of contigs to be conisdered (after -M) .....: 1,990,377 Number of splits ..................................: 2,014,055 Number of nucleotides .............................: 5,732,881,110

✖ anvi-profile encountered an error after 0:06:53.690773 Traceback (most recent call last): File "/apps/anaconda3-2023.03/envs/anvio-8/bin/anvi-profile", line 130, in main(args) File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/anvio/terminal.py", line 915, in wrapper program_method(*args, **kwargs) File "/apps/anaconda3-2023.03/envs/anvio-8/bin/anvi-profile", line 39, in main profiler.BAMProfiler(args)._run() File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/anvio/profiler.py", line 581, in _run self.profile_multi_thread() File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/anvio/profiler.py", line 1169, in profile_multi_thread proc.start() File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/process.py", line 121, in start self._popen = self._Popen(self) File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/context.py", line 281, in _Popen return Popen(process_obj) File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/popen_fork.py", line 19, in init self._launch(process_obj) File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/popen_fork.py", line 66, in _launch self.pid = os.fork() OSError: [Errno 12] Cannot allocate memory Process Process-2: Traceback (most recent call last): File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/managers.py", line 810, in _callmethod conn = self._tls.connection AttributeError: 'ForkAwareLocal' object has no attribute 'connection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/anvio/profiler.py", line 948, in profile_contig_worker index = available_index_queue.get(True) File "", line 2, in get File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/managers.py", line 814, in _callmethod self._connect() File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/managers.py", line 801, in _connect conn = self._Client(self._token.address, authkey=self._authkey) File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/connection.py", line 505, in Client c = SocketClient(address) File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/connection.py", line 633, in SocketClient s.connect(address) FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/managers.py", line 810, in _callmethod conn = self._tls.connection AttributeError: 'ForkAwareLocal' object has no attribute 'connection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap self.run() File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/anvio/profiler.py", line 967, in profile_contig_worker output_queue.put(e) File "", line 2, in put File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/managers.py", line 814, in _callmethod self._connect() File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/managers.py", line 801, in _connect conn = self._Client(self._token.address, authkey=self._authkey) File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/connection.py", line 505, in Client c = SocketClient(address) File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/connection.py", line 633, in SocketClient s.connect(address) FileNotFoundError: [Errno 2] No such file or directory Process Process-3: Traceback (most recent call last): File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/managers.py", line 810, in _callmethod conn = self._tls.connection AttributeError: 'ForkAwareLocal' object has no attribute 'connection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/anvio/profiler.py", line 948, in profile_contig_worker index = available_index_queue.get(True) File "", line 2, in get File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/managers.py", line 814, in _callmethod self._connect() File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/managers.py", line 801, in _connect conn = self._Client(self._token.address, authkey=self._authkey) File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/connection.py", line 505, in Client c = SocketClient(address) File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/connection.py", line 633, in SocketClient s.connect(address) FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/managers.py", line 810, in _callmethod conn = self._tls.connection AttributeError: 'ForkAwareLocal' object has no attribute 'connection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap self.run() File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/anvio/profiler.py", line 967, in profile_contig_worker output_queue.put(e) File "", line 2, in put File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/managers.py", line 814, in _callmethod self._connect() File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/managers.py", line 801, in _connect conn = self._Client(self._token.address, authkey=self._authkey) File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/connection.py", line 505, in Client c = SocketClient(address) File "/apps/anaconda3-2023.03/envs/anvio-8/lib/python3.10/site-packages/multiprocess/connection.py", line 633, in SocketClient s.connect(address) FileNotFoundError: [Errno 2] No such file or directory

meren commented 1 month ago

Hi @kinosham,

This is what anvi-profile says:

OSError: [Errno 12] Cannot allocate memory

You are running out of memory. Please contact your sys-admin / HPC personnel for guidance.

Best wishes, Meren