ncbi / egapx

Eukaryotic Genome Annotation Pipeline-External caller scripts and documentation
Other
19 stars 1 forks source link

Error executing process #14

Open znelson999 opened 1 month ago

znelson999 commented 1 month ago

Hello Again,

I'm attempting to run EGAP on a new assembled genome and am getting a new error. Attached will be the code submitted via shell script and a portion of the slurm file output. This new genome is not hosted in NCBI, it is a local .fasta file that is roughly 5 times the size (2.1 gigabytes) of the other assembly I was working with in a previous issue. There is no transcriptome data included this time.

Let me know if you need anything else from me.

EgapxGSIAttempt1.txt slurm-12686201.txt

victzh commented 1 month ago

You seem to have a problem with incorrect docker/singularity image. Can you look into /project/cricket_gen/ZachN/Virtual/WorkingDirectory/singularity, there should be a file ncbi_egapx.img. If it's there, can you try singularity shell /project/cricket_gen/ZachN/Virtual/WorkingDirectory/singularity/ncbi_egapx.img You can try to rm -rf /project/cricket_gen/ZachN/Virtual/WorkingDirectory/singularity to clear the cached image.

Why when you run with -e singularity you get SLURM output file? You should use -e slurm and modify slurm.config file to tune it to your SLURM configuration and enable singularity in it.

After the first run there is a directory created for you by EGAPx - /project/cricket_gen/ZachN/egapx_config and there are several config files roughly corresponding to Nextflow 'executors'. If you run on SLURM using Singularity, you need to edit the file /project/cricket_gen/ZachN/egapx_config/slurm.config and replace /data/$USER with your network accessible space, probably /project/cricket_gen/ZachN. It will create extra 2 directories - tmp and singularity in it. You may need to add some other parameters specific to your SLURM/Singularity installation. If you have publicly available documentation on your particular installation I can look into it.

znelson999 commented 4 weeks ago

Hello,

In the previous version I had to use the -e singularity to run EGAPx; there didn't seem to be a -e slurm parameter (I could have missed it) and I didn't change the code between versions thinking it would be fine. Everything is submitted to the cluster through .sh files and have always generated a slurm output file. That's the only way I know.

Regardless, I've switched the -e parameter and our HPC has installed EGAPx and have run it through module load and submitting by .sh which fixed some problems. It is still failing, attached is the tail end of the slurm output file.

slurmout.txt

Any ideas why this could be happening?

Thank you for the assistance and I appreciate any correction or education you can provide.

(Edited with correction.)

pstrope commented 3 weeks ago

Hi, It looks like egapx is using an older image from the cache. Did you try rm -rf /project/cricket_gen/ZachN/Virtual/WorkingDirectory/singularity to clear the cached image? Path to the singularity dir could be different than this.

pstrope commented 3 weeks ago

And make sure you have the latest scripts from github (v0.1.2-alpha).

pstrope commented 2 weeks ago

What is the taxid you are using?