bcgsc / LINKS

⛓ Long Interval Nucleotide K-mer Scaffolder
GNU General Public License v3.0
73 stars 15 forks source link

Invalid format of "-s file-of-filenames" #64

Closed Hai1983 closed 2 years ago

Hai1983 commented 2 years ago

Hi Sir/Madam.

I try to run LINKS with :(_/LINKS/bin/./LINKS -f Catfishpilon2/pilon.fasta -s Hifi.fof -b Catfish -d 1000,2000,3000 -t 10 -k 15 -l 5 -a 0.3) But face the issue ; __"[2021-12-18 01:46:56]ESC[31m[ERROR] ESC[0mHiFi.txt source file is in invalid format! make: * [/home/users/d/a/daomhai/Linktest/LINKS/bin/./LINKS-make:131: Catfish.tigpair_checkpoint.tsv] Error 1** (END)"__

The (Hifi.txt/Hifi.fa ) was generated from : gunzip -c HiFi_Q20.fastq.gz | perl -ne '$ct++;if($ct>4){$ct=1;}print if($ct<3);' > HiFi.fa echo Hifi.fa > Hifi.fof, (echo Hifi.txt > Hifi.fof). I try run two times with each HiFi.fa and Hifi.txt and got the same problem.

Please help me to figure out what I am doing wrong !!

Sincerely, Hai

MurathanGoktas commented 2 years ago

Hello,

Please try echo HiFi_Q20.fastq.gz > reads.fof and provide it as -s reads.fof.

LINKS can read .fastq.gz.

Let us know if you have any other question.

Hai1983 commented 2 years ago

Dear Murathan

Thanks for your advice. This issue is Ok now. However, I face with the new problem for running. I used the cluster for analying . _(Writing Bloom filter to disk () : 1639825641 Done mybf, printing stats... 240086 ms Read fasta stage 100000 reads processed 200000 reads processed make: * [/home/users/d/a/daomhai/Linktest/LINKS/bin/./LINKS-make:131: Catfish.tigpair_checkpoint.tsv] Killed slurmstepd: error: Detected 1 oom-kill event(s) in step 2088635.batch cgroup. Some of your processes may have been killed by the cgroup out-of-memory handler**.)_

My job is ; ( _#!/bin/bash

SBATCH --time=10:00:00 # hh:mm:ss

SBATCH --ntasks=1

SBATCH --cpus-per-task=16

SBATCH --mem-per-cpu=4000 # megabytes

module load Perl/5.30.0-GCCcore-8.3.0

module load GCC/8.3.0

/home/users/d/a/daomhai/Linktest/LINKS/bin/./LINKS -f /scratch/users/d/a/daomhai/shortrundata/350-550TT/BWA/catfish_pilon_1/Catfishpilon2/pilon.fasta -s reads.fof -b Catfish -d 1000,2000,3000 -t 10 -k 15 -l 5 -a 0.3 )

Please give me some advice on this issue. Sincerely, Hai

MurathanGoktas commented 2 years ago

Hi,

This error is not related with LINKS, you hit the maximum requested memory for this job. Try increasing the requested memory in your cluster for this job.

Hai1983 commented 2 years ago

Dear Murathan

Thanks for great help.

Now I try to increase the memory for my job (#SBATCH --mem-per-cpu=100000 # megabytes). But they still fail! Do you have some advice on "-d and -t " for minimize memory usage ???. ( Now I try with -d 5000 and -t 200) My Hifi.fasta file (~ 17Gb) and my draft genome (~7,8 Mb). May I reduce the size of Hifi.fasta file (by spliting to smaller size) to reduce the memory usage ?? Hvae you other suggestion for me.??

Thank you in advance . Sincerely, Hai

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your interest in LINKS!