AGAT takes >12 hours on some GFFs on HPC

[chriswyatt1]

Hi,

First, thanks for maintaining a great tool. I think maybe I am doing something wrong, but have been using the "agat_sp_keep_longest_isoform.pl" tool, and it finishes 95% of the time within a few minutes, but some gffs are taking more than 12 hours. Maybe this is the expected behaviour, so may not be an issue as such.

For example with Bombus terrestris (~14 hours, once job is running [its not stuck in a hpc queue]): agat_sp_keep_longest_isoform.pl -gff Bombus_terrestris-GCA_910591885.2-2022_03-genes.gff3.gz -o Bombus_terrestris.longest.gff3

The gff can be downloaded here: GFF3

General (please complete the following information):

• AGAT version : biocontainers/agat:1.3.0--pl5321hdfd78af_0
• AGAT installation/use biocontainers/agat:1.3.0--pl5321hdfd78af_0 (singularity on sge HPC)
• OS: Red Hat Enterprise Linux

To Reproduce The script I run is as follows: agat_sp_keep_longest_isoform.pl -gff Bombus_terrestris-GCA_910591885.2-2022_03-genes.gff3.gz -o Bombus_terrestris.longest.gff3 This is my output: out.txt

[Juke34]

In a standalone way on my laptop your file took ~1 min. So I guess the problem is related to your infrastructure. It might be due to RAM unavailable, AGAT load everything in RAM to work efficiently, if no RAM is available the computer will use the SWAP (HDD) and that can be very slow. It can be also related due concurrency with other job... if CPU is overloaded by different task, or it might be slows down by intense IO if e.g. there is a backup on the system in the same time in background.

[chriswyatt1]

Ahhh, ok great. I will check out if I can increase the RAM to get this job through. Thanks for your help!

[Juke34]

To be on the safe side it is good to take 10 times the size of the file. So for a file of 100mb, it is good to have 1GB RAM available.