MGEScan / mgescan

A Galaxy based system for identifying retrotransposons in genome
http://mgescan.github.io/mgescan/
GNU General Public License v3.0
5 stars 5 forks source link

Error in tempfile() using template #19

Open intirules opened 6 years ago

intirules commented 6 years ago

Im running MGESCAN in my University server:

Filesystem Size Used Avail Use% Mounted on /dev/sda4 875G 78G 753G 10% / devtmpfs 252G 0 252G 0% /dev tmpfs 252G 0 252G 0% /dev/shm tmpfs 252G 706M 252G 1% /run tmpfs 252G 0 252G 0% /sys/fs/cgroup /dev/sda2 968M 116M 787M 13% /boot /dev/mapper/vg_data-data 1.8T 77M 1.7T 1% /data /dev/sda1 198M 9.5M 188M 5% /boot/efi hn01:/opt/hpe/hpc/shared 717G 101G 580G 15% /opt/hpe/hpc/shared hn01:/shared 717G 101G 580G 15% /shared hn01:/opt/yum/rhel7.3 717G 101G 580G 15% /opt/yum/rhel7.3 sgclnas01:/ifs/HPC-Cluster_NFS/Medicina 8.0T 2.2T 5.9T 27% /medicina sgclnas01:/ifs/HPC-Cluster_NFS/Finanzas 3.0T 7.7G 3.0T 1% /finanzas sgclnas01:/ifs/HPC-Cluster_NFS/Sistemas 3.0T 0 3.0T 0% /sistemas sgclnas01:/ifs/home 100G 46G 55G 46% /home tmpfs 51G 0 51G 0% /run/user/0 tmpfs 51G 0 51G 0% /run/user/2003

so i can run it simultaneously to 22 genomes, them are distributed in 22 folders to get output in this folders for the next steps. Im using this script: "$CONTADOR"_gt.sh sbatch -N 1 -n 1 -o "$CONTADOR"_gt.out --partition=Bioinfo "$CONTADOR"_gt.sh

echo -e '#!/bin/bash\ncd /medicina/wocana/Tesis/Secuencias/MGES/'"$CONTADOR"'\n' > "$CONTADOR"_ltr.sh echo -e 'module load anaconda/2.7\nsource ~/.mgescanrc\n' >> "$CONTADOR"_ltr.sh echo -e 'mgescan ltr /medicina/wocana/Tesis/Secuencias/MGES/'"$CONTADOR"' --output=/medicina/wocana/Tesis/Secuencias/MGES/'"$CONTADOR"'/Salida\n' >> "$CONTADOR"_ltr.sh sbatch -N 1 -n 1 -o "$CONTADOR"_ltr.out --partition=Bioinfo "$CONTADOR"_ltr.sh done

The program was running normaly, it still going with two of the 22 genomes. The other 20 i got this ERROR message. We already check and its not possible that the memory were full im not sure if i made a mistake on the MGESCAN parameters, i already check and theres no option to have multiple process at once.

Making bin Finding putative ltr 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000 110000 120000 130000 140000 150000 160000 170000 180000 190000 200000 210000 220000 230000 240000 250000 260000 270000 280000 290000 300000 310000 320000 330000 340000 350000 360000 370000 380000 390000 400000 410000 420000 430000 440000 450000 460000 470000 480000 490000 Finding LTRs Finding MEM Making bin Error in tempfile() using template /tmp/XXXXXXXXXX.temp3: Could not create temp file /tmp/ZzhAxkjC82.temp3: No space left on device at /usr/local/Bioinfo/mgescan3/src/mgescan/ltr/find_ltr.pl line 1256. Finding putative ltr Process Process-1: Traceback (most recent call last): File "/usr/local/anaconda3/envs/py27/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap self.run() File "/usr/local/anaconda3/envs/py27/lib/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "build/bdist.linux-x86_64/egg/mgescan/cmd.py", line 178, in ltr res1 = self.run_cmd(cmd1) File "build/bdist.linux-x86_64/egg/mgescan/cmd.py", line 225, in run_cmd retcode = check_call(cmd.split()) File "/usr/local/anaconda3/envs/py27/lib/python2.7/subprocess.py", line 186, in check_call raise CalledProcessError(retcode, cmd) CalledProcessError: Command '['/usr/local/Bioinfo/mgescan3/src/mgescan/ltr/find_ltr.pl', '-genome=/medicina/wocana/Tesis/Secuencias/MGES/3/divided-genome', '-data=/medicina/wocana/Tesis/Secuencias/MGES/3/Salida', '-hmmerv=3', '-min_dist=2000', '-max_dist=20000', '-min_len_ltr=130', '-max_len_ltr=2000', '-ltr_sim_condition=70', '-cluster_sim_condition=70', '-len_condition=70']' returned non-zero exit status 28 ltr: starting

lee212 commented 6 years ago

Check and clean /tmp/ directory, if it's full. I find the error message: No space left on device from your message. Please tell me more, if you have further issues.

intirules commented 6 years ago

Hi we already check and its not possible that the device is full. We have 52 Gb for that. Two questions: 1) Its there a way to redirect tmp-files to an specific folder? 2) its possible to retake from the error-point a process?

intirules commented 6 years ago

The thing is that we wanna run as many genomes simultaneusly as possible. We already check and the programm at the momment creates just for 1 genome 8.8 Gb. As i said we have just 52 Gb in our tmp folder (Server). We would like to redirect tmp files to another repository were we have 6TB. Its possible ?

lee212 commented 6 years ago

It requires re-compiling and re-installing python library for specifying a temp directory in your favor. I just started to add --temp_dir= optional parameter under temp_dir branch (see https://github.com/MGEScan/mgescan/tree/temp_dir). If you can upgrade or re install mgescan package on your system with this development branch, you probably be able to change the temp directory with the new parameter, for example, mgescan both /home/user1/sequence --temp_dir=/my_new_temp_directory