search

nextgenusfs / funannotate

Eukaryotic Genome Annotation Pipeline
http://funannotate.readthedocs.io
BSD 2-Clause "Simplified" License
300 stars 82 forks source link

CMD ERROR: CodingQuarry #807

Open Rahimlou opened 1 year ago

Rahimlou commented 1 year ago

I'm trying to run Funannotate Predict with a genome (fasta), trinity assembly of transcriptome (fasta), and alignment (bam) input files. I encounter the following error: CMD ERROR: CodingQuarry -p 2 -f /gpfs/space/home/barabi/Myers_Project/Gene_Prediction/predict_misc/genome.softmasked.fa -t /gpfs/space/home/barabi/Myers_Project/Gene_Prediction/predict_misc/stringtie.gff3

I have those input files for CodingQuarry generated in previous steps. Please let me know if you need more information.

hyphaltip commented 1 year ago

Are there error messages in logfile in the logfiles folder?

Does the funannotate test work for coding quarry?

Rahimlou commented 1 year ago

funannotate test worked for codingquarry. I have the full thing installed.

This is the only error I got in funannotate-predict.log file.

I used the following code to align my transcriptome reads to the genome which might help you in finding the problem:

minimap2 -ax sr \ -t 16 /gpfs/space/home/barabi/Myers_Project/Allomyces_arbuscula_Burma_1F.masked.fasta \ /gpfs/space/home/barabi/Myers_Project/transcriptome_raw_reads_myc/6413-JM-1_qfilter_R1.fasta /gpfs/space/home/barabi/Myers_Project/transcriptome_raw_reads_myc/6413-JM-1_qfilter_R2.fasta \ | samtools sort -@16 -O BAM -o ASSEMBLY_NAME.DRR008460.bam -

Rahimlou commented 1 year ago

I tried to run CodingQuarry individually using the same input and I got the following error: Linux ares11 3.10.0-1160.42.2.el7.x86_64 #1 SMP Tue Sep 7 14:49:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux /var/spool/slurm/slurmd/job33790697/slurm_script: line 24: 182980 Segmentation fault (core dumped) CodingQuarry -p 2 -f /gpfs/space/home/barabi/Myers_Project/Gene_Prediction/predict_misc/genome.softmasked.fa -t /gpfs/space/home/barabi/Myers_Project/Gene_Prediction/predict_misc/stringtie.gff3

Rahimlou commented 1 year ago

Are there error messages in logfile in the logfiles folder?

Does the funannotate test work for coding quarry?

funannotate test worked for codingquarry. I have the full thing installed.

This is the only error I got in funannotate-predict.log file.

I used the following code to align my transcriptome reads to the genome which might help you in finding the problem:

minimap2 -ax sr -t 16 /gpfs/space/home/barabi/Myers_Project/Allomyces_arbuscula_Burma_1F.masked.fasta /gpfs/space/home/barabi/Myers_Project/transcriptome_raw_reads_myc/6413-JM-1_qfilter_R1.fasta /gpfs/space/home/barabi/Myers_Project/transcriptome_raw_reads_myc/6413-JM-1_qfilter_R2.fasta | samtools sort -@16 -O BAM -o ASSEMBLY_NAME.DRR008460.bam -

Rahimlou commented 1 year ago

I tried to run CodingQuarry individually using the same input and I got the following error: Linux ares11 3.10.0-1160.42.2.el7.x86_64 #1 SMP Tue Sep 7 14:49:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux /var/spool/slurm/slurmd/job33790697/slurm_script: line 24: 182980 Segmentation fault (core dumped) CodingQuarry -p 2 -f /gpfs/space/home/barabi/Myers_Project/Gene_Prediction/predict_misc/genome.softmasked.fa -t /gpfs/space/home/barabi/Myers_Project/Gene_Prediction/predict_misc/stringtie.gff3

I tried to run CodingQuarry individually using the same input and I got the following error: Linux ares11 3.10.0-1160.42.2.el7.x86_64 https://github.com/nextgenusfs/funannotate/issues/1 SMP Tue Sep 7 14:49:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux /var/spool/slurm/slurmd/job33790697/slurm_script: line 24: 182980 Segmentation fault (core dumped) CodingQuarry -p 2 -f /gpfs/space/home/barabi/Myers_Project/Gene_Prediction/predict_misc/genome.softmasked.fa -t /gpfs/space/home/barabi/Myers_Project/Gene_Prediction/predict_misc/stringtie.gff3

skylerhar1 commented 1 year ago

I am running into this same issue where I get a CMD Error through Funannotate and a seg fault when I run CodingQuarry individually.

lalalagartija commented 1 year ago

Same here ! Another issue says that we can avoid it by applying : --weights codingquarry:0. It worked but rendered very poor results.

I have also installed CodingQuarry manually and it runs well. Now the question is how to make funannotate use the manually installed CodingQuarry with singularity

lalalagartija commented 1 year ago

This version of CodingQuarry works : https://hub.docker.com/r/robsyme/codingquarry/dockerfile

MichaelFokinNZ commented 6 months ago

This version of CodingQuarry works : https://hub.docker.com/r/robsyme/codingquarry/dockerfile @lalalagartija have you managed to get this version of coding quarry to your funannotate singularity container?

xiaojingshi commented 5 months ago

I tried to run CodingQuarry individually using the same input and I got the following error: Linux ares11 3.10.0-1160.42.2.el7.x86_64 #1 SMP Tue Sep 7 14:49:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux /var/spool/slurm/slurmd/job33790697/slurm_script: line 24: 182980 Segmentation fault (core dumped) CodingQuarry -p 2 -f /gpfs/space/home/barabi/Myers_Project/Gene_Prediction/predict_misc/genome.softmasked.fa -t /gpfs/space/home/barabi/Myers_Project/Gene_Prediction/predict_misc/stringtie.gff3

I tried to run CodingQuarry individually using the same input and I got the following error: Linux ares11 3.10.0-1160.42.2.el7.x86_64 #1 SMP Tue Sep 7 14:49:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux /var/spool/slurm/slurmd/job33790697/slurm_script: line 24: 182980 Segmentation fault (core dumped) CodingQuarry -p 2 -f /gpfs/space/home/barabi/Myers_Project/Gene_Prediction/predict_misc/genome.softmasked.fa -t /gpfs/space/home/barabi/Myers_Project/Gene_Prediction/predict_misc/stringtie.gff3

I got exactly the same result. How did you solve this?

calizilla commented 2 months ago

Thank you @lalalagartija that docker container is the saviour! I have a conda installation of funannotate that seg faults at CodingQuarry, a manually installed CodingQuarry that also seg faults, but running this image after pulling with singularity works great.

I ran the CodingQuarry step manually, which created the GFF3 ./out/PredictedPass.gff3. Funanotate reformats this GFF3 and renames it to ./predict_misc/coding_quarry.gff3 - see below example - so if anyone wanted to run the CodingQuarry step manually and then add it back in to the funannotate output directory so that the predict step could be resumed and find the existing CodingQuarry GFF3, make sure to identify which part of library.py or predict.py is performing the GFF3 reformatting and run that also. 

[funannotate_output]$ head predict_misc/coding_quarry.gff3 
##gff-version 3
scaffold_100A   CodingQuarry    gene    11120   11446   .       +       .       ID=gene_1;Name=gene_1;Alias=NS.00001;
scaffold_100A   CodingQuarry    mRNA    11120   11446   .       +       .       ID=transcript_1-T1;Parent=gene_1;Alias=NS.00001;
scaffold_100A   CodingQuarry    exon    11120   11446   .       +       .,      ID=transcript_1-T1.exon1;Parent=transcript_1-T1;
scaffold_100A   CodingQuarry    CDS     11120   11446   .       +       0       ID=transcript_1-T1.cds;Parent=transcript_1-T1;
scaffold_100A   CodingQuarry    gene    14368   17277   .       -       .       ID=gene_2;Name=gene_2;Alias=NS.00002;
scaffold_100A   CodingQuarry    mRNA    14368   17277   .       -       .       ID=transcript_2-T1;Parent=gene_2;Alias=NS.00002;
scaffold_100A   CodingQuarry    exon    14368   16752   .       -       .,      ID=transcript_2-T1.exon1;Parent=transcript_2-T1;
scaffold_100A   CodingQuarry    CDS     14368   16752   .       -       0       ID=transcript_2-T1.cds;Parent=transcript_2-T1;
scaffold_100A   CodingQuarry    exon    16800   16818   .       -       .,      ID=transcript_2-T1.exon2;Parent=transcript_2-T1;
[ref]$ head ./out/PredictedPass.gff3 
scaffold_100A   CodingQuarry_v1.1       gene    11120   11446   .       +       .       ID=NS.00001;Name=;
scaffold_100A   CodingQuarry_v1.1       CDS     11120   11446   .       +       0       ID=CDS:NS.00001;Parent=NS.00001;
scaffold_100A   CodingQuarry_v1.1       gene    14368   17277   .       -       .       ID=NS.00002;Name=;
scaffold_100A   CodingQuarry_v1.1       CDS     14368   16752   .       -       0       ID=CDS:NS.00002;Parent=NS.00002;
scaffold_100A   CodingQuarry_v1.1       CDS     16800   16818   .       -       0       ID=CDS:NS.00002;Parent=NS.00002;
scaffold_100A   CodingQuarry_v1.1       CDS     16869   16945   .       -       2       ID=CDS:NS.00002;Parent=NS.00002;
scaffold_100A   CodingQuarry_v1.1       CDS     17008   17066   .       -       0       ID=CDS:NS.00002;Parent=NS.00002;
scaffold_100A   CodingQuarry_v1.1       CDS     17122   17162   .       -       1       ID=CDS:NS.00002;Parent=NS.00002;
scaffold_100A   CodingQuarry_v1.1       CDS     17217   17232   .       -       2       ID=CDS:NS.00002;Parent=NS.00002;
scaffold_100A   CodingQuarry_v1.1       CDS     17253   17277   .       -       1       ID=CDS:NS.00002;Parent=NS.00002;

@MichaelFokinNZ - you can add the singularity run command to the funannotate library.py script. Note that I also added the -d flag to CodingQuarry as I have unstranded RNAseq: 

  6321          "singularity", "run", "codingquarry_latest.sif",
  6322          "CodingQuarry",
  6323          "-d"
  6324          "-p",
  6325          str(cpus),
  6326          "-f",
  6327          os.path.realpath(genome),
  6328          "-t",
  6329          os.path.realpath(stringtieGFF3),
  6330      ]

And likewise for the second CodingQuarry command a few lines further down in the script.

dingcaojin commented 1 month ago

@calizilla Hi, I have encountered the same problem. Howerver, I can't pulling robsyme/codingquarry by docker in my wsl2. Is my code have mistake? docker pull robsyme/codingquarry:latest latest: Pulling from robsyme/codingquarry [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of docker.io/robsyme/codingquarry:latest to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/ docker run robsyme/codingquarry Unable to find image 'robsyme/codingquarry:latest' locally latest: Pulling from robsyme/codingquarry docker: [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of docker.io/robsyme/codingquarry:latest to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/. I also tried others tags of robsyme/codingquarry.

calizilla commented 1 month ago

Hi @dingcaojin - I don't use docker as it's not permitted on the cluster I work on. I used the below command to pull the docker file as a singularity image:

singularity pull docker://robsyme/codingquarry:latest

Are you able to install singularity on your wsl2? It's a pretty straightforward install usually.

Once you've pulled the image with singularity, test it works before modifying your funannotate library.py:

singularity run codingquarry_latest.sif CodingQuarry -h
dingcaojin commented 1 month ago

Hi @dingcaojin - I don't use docker as it's not permitted on the cluster I work on. I used the below command to pull the docker file as a singularity image:

singularity pull docker://robsyme/codingquarry:latest

Are you able to install singularity on your wsl2? It's a pretty straightforward install usually.

Once you've pulled the image with singularity, test it works before modifying your funannotate library.py:

singularity run codingquarry_latest.sif CodingQuarry -h

YES! Singularity could working on my wsl2, and I finally got the results of Codingquarry. Thanks you very much. Your help is very effective. I will follow you.

calizilla commented 1 month ago

@dingcaojin I'm happy to have helped :-) Note that if you run CodingQuarry manually, outside of funannotate instead of modifying library.py to run it with sungularity, to make the adjustments described earlier (change gff3 format and filepath) to match what funannotate expects, so that funannotate can find the CodingQuarry output and continue with the existing output, rather than trying to run it again.

nextgenusfs commented 1 month ago

Nice job all. Note that you shouldn't need to modify the funannotate source code (I hope), but rather you could wrap a shell script around your docker/singularity builds and then put them in the PATH so that funannotate picks them up, ie that's what I do locally with genemark as it won't run on my Mac: https://github.com/nextgenusfs/dockerized-genemark