augustus: ERROR PP::Profile: Error parsing pattern file

[aersoares81]

I'm using the latest Funnanotate, but when I reach the "funannotate predict" step I get this:

augustus: ERROR
        PP::Profile: Error parsing pattern file"/sw/anaconda/2019.10/envs/funannotate-test/lib/python3.8/site-packages/funannotate/config/EOG092C0B3U.prfl", line 8.
[May 18 12:01 PM]: ERROR: augustus --proteinprofile test failed, likely a compilation error. This is required to run BUSCO, exiting.

(conda env is called funannotate-test)

which Augustus give me: /sw/anaconda/2019.10/envs/funannotate-test/bin/augustus So I think it's looking at the right Augustus. I tried updating Augustus, but

Updating augustus is constricted by
/ funannotate -> requires augustus==3.3.3=pl5262h5a9fe7b_6

"

So I did a "conda install augustus==3.3.3=pl5262h5a9fe7b_6 --force-reinstall" just in case, but the error persists. (curiously I have another instance of funannotate running and it worked despite some augustus errors. Also, maybe it's worth noting, that I re-installed it fresh on this new conda environment because it was the only way to go around a "Diamond repeat database was created with different version of diamond, please re-run funannotate setup" error message that never disappeared no matter what I did.)

Any suggestions?

[Midreancgle]

I have the same problem

[nextgenusfs]

This is a compilation issue with Augustus with your system. This method is required for BUSCO to work. What OS are you on? Probably need to try a different conda build I guess or compile/install manually.

[Midreancgle]

My system is Ubuntu. The funannotate had been working for a while, and then it didn't work with following problems:

(funannotate) + [ /home/yue/anaconda3/envs/funannotate/test] funannotate predict -i GCA 000002515. 1 ASM251v1_genomic. fna -o . / species "Kluyveromyces Iactis" - - cpus 45 . May 20 12: 03 PM] : OS: Ubuntu 18. 04, 56 cores, ~ 264 GB RAM. Python: 3. 8.12 [May2012:03PM]:Running funannotate v1.8.9 Traceback ( most recent call Iast) : Fi I e " /home/yue/anaconda3/envs/funannotate/bin/funannotate", 1i ne 10, i n sys. exit( main( )) Fi I e " /home/yue/anaconda3/envs/funannotate/lib/python3. 8/site-packages/funannotate/funannotate. py", Iine 705, in main mod. mai n( arguments) File " /home/yue/anaconda3/envs/funannotate/lib/python3. 8/site-packages/funannotate/predict. py", Iine 472, in main forfinos.listdir(os.path.join(LOCALAUGUSTUS,'species',augspecies)): FiIeNotFoundError: [ Errno 2] No such fiIe or directory: '. /predict misc/ab_initio_parameters/augustus/species/kI uyveromyces_lactis'

I reinstalled it and I had this compile problem.

[zsdxgl]

I have the same problem

[hyphaltip]

I have the same problem the prediction folder failing? -what folders are created in the directory? Does predict_misc get created.

@Midreancgle the spaces in your quoted errors are hard to read - can you put these in direct quote in github eg (in here)

[mgarl-10]

Same problem here

[katajar]

Same here. In my case test failed.

augustus: ERROR
    PP::Profile: Error parsing pattern file"/home/agazar/anaconda3/envs/funannotate/lib/python3.7/site-packages/funannotate/config/EOG092C0B3U.prfl", line 8.
[May 24 01:10 PM]: ERROR: augustus --proteinprofile test failed, likely a compilation error. This is required to run BUSCO, exiting.

[nextgenusfs]

In the docker image I can't reproduce (my dev machine is Mac so only linux test I have is docker), it installed this version of augustus (so same as yours @aersoares81):

 augustus                                3.3.3  pl5262h5a9fe7b_6     bioconda/linux-64         77 MB

The docker image is running Debian (so not Ubuntu, although ubuntu based on Debian). Here is the successful test:

$ funannotate_dev/funannotate-docker test -t busco --cpus 4
#########################################################
Running `funannotate predict` BUSCO-mediated training unit testing
CMD: funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o annotate --cpus 4 --species Awesome busco
#########################################################
-------------------------------------------------------
[May 24 08:14 AM]: OS: Debian GNU/Linux 10, 4 cores, ~ 8 GB RAM. Python: 3.8.12
[May 24 08:14 AM]: Running funannotate v1.8.10
[May 24 08:14 AM]: GeneMark not found and $GENEMARK_PATH environmental variable missing. Will skip GeneMark ab-initio prediction.
[May 24 08:14 AM]: Skipping CodingQuarry as no --rna_bam passed
[May 24 08:14 AM]: Parsed training data, run ab-initio gene predictors as follows:
  Program      Training-Method
  augustus     busco          
  glimmerhmm   busco          
  snap         busco          
[May 24 08:14 AM]: Loading genome assembly and parsing soft-masked repetitive sequences
[May 24 08:14 AM]: Genome loaded: 6 scaffolds; 3,776,588 bp; 19.75% repeats masked
[May 24 08:14 AM]: Mapping 1,065 proteins to genome using diamond and exonerate
[May 24 08:14 AM]: Found 1,505 preliminary alignments with diamond in 0:00:02 --> generated FASTA files for exonerate in 0:00:00
[May 24 08:15 AM]: Exonerate finished in 0:00:39: found 1,270 alignments
[May 24 08:15 AM]: Running BUSCO to find conserved gene models for training ab-initio predictors
[May 24 08:30 AM]: 373 valid BUSCO predictions found, validating protein sequences
[May 24 08:31 AM]: 370 BUSCO predictions validated
[May 24 08:31 AM]: Training Augustus using BUSCO gene models
[May 24 08:31 AM]: Augustus initial training results:
  Feature       Specificity   Sensitivity
  nucleotides   99.5%         83.8%      
  exons         71.8%         59.7%      
  genes         86.5%         59.3%      
[May 24 08:31 AM]: Running Augustus gene prediction using awesome_busco parameters
[May 24 08:32 AM]: 1,301 predictions from Augustus
[May 24 08:32 AM]: Pulling out high quality Augustus predictions
[May 24 08:32 AM]: Found 313 high quality predictions from Augustus (>90% exon evidence)
[May 24 08:32 AM]: Running SNAP gene prediction, using training data: annotate/predict_misc/busco.final.gff3
[May 24 08:35 AM]: 1,529 predictions from SNAP
[May 24 08:35 AM]: Running GlimmerHMM gene prediction, using training data: annotate/predict_misc/busco.final.gff3
[May 24 08:39 AM]: 1,769 predictions from GlimmerHMM
[May 24 08:39 AM]: Summary of gene models passed to EVM (weights):
  Source         Weight   Count
  Augustus       1        988  
  Augustus HiQ   2        313  
  GlimmerHMM     1        1769 
  snap           1        1529 
  Total          -        4599 
[May 24 08:39 AM]: EVM: partitioning input to ~ 35 genes per partition using min 1500 bp interval
[May 24 08:50 AM]: Converting to GFF3 and collecting all EVM results
[May 24 08:50 AM]: 1,691 total gene models from EVM
[May 24 08:50 AM]: Generating protein fasta files from 1,691 EVM models
[May 24 08:50 AM]: now filtering out bad gene models (< 50 aa in length, transposable elements, etc).
[May 24 08:50 AM]: Found 143 gene models to remove: 0 too short; 0 span gaps; 143 transposable elements
[May 24 08:50 AM]: 1,548 gene models remaining
[May 24 08:50 AM]: Predicting tRNAs
[May 24 08:51 AM]: 112 tRNAscan models are valid (non-overlapping)
[May 24 08:51 AM]: Generating GenBank tbl annotation file
[May 24 08:51 AM]: Collecting final annotation files for 1,660 total gene models
[May 24 08:51 AM]: Converting to final Genbank format
[May 24 08:51 AM]: Funannotate predict is finished, output files are in the annotate/predict_results folder
[May 24 08:51 AM]: Your next step might be functional annotation, suggested commands:
-------------------------------------------------------
Run InterProScan (manual install): 
funannotate iprscan -i annotate -c 4

Run antiSMASH (optional): 
funannotate remote -i annotate -m antismash -e youremail@server.edu

Annotate Genome: 
funannotate annotate -i annotate --cpus 4 --sbt yourSBTfile.txt
-------------------------------------------------------

[May 24 08:51 AM]: Training parameters file saved: annotate/predict_results/awesome_busco.parameters.json
[May 24 08:51 AM]: Add species parameters to database:

  funannotate species -s awesome_busco -a annotate/predict_results/awesome_busco.parameters.json

#########################################################
SUCCESS: `funannotate predict` BUSCO-mediated training test complete.
#########################################################

So you can probably help each other by sharing the augustus builds and which are failing on which OS, etc -- and you can test your installation with funannotate test -t busco --cpus x. If you get the PP::Profile error above it means Augustus isn't compiled properly. This is an issue that goes back at least 4-5 years that I've raised but it seems to get fixed and then broken again depending on how augustus is compiled, on macOS the only way I could get it to work was to use a specific version (3.2.1) and modify the compilation. For linux it has generally worked from bioconda over the years, but sometimes builds are incompatible with some OS, so try a different build if possible otherwise just download source and compile/install manually and remove the bioconda installed augustus. Use of funannotate test can help you troubleshoot the install, but this is an augustus install issue and not something I can fix. I have on my list to make augustus >3.4 compatible, but currently that still does not work with funannotate.

[aersoares81]

Following your comment I decided to test run it via Docker on my local macOS machine. I installed Funannotate following the "Quickest start Docker" instructions. This local machine never had any version of augustus, funannotate, etc, before (macOS v.11.6.4). This is what I got:

funannotate-docker test -t predict --cpus 8
#########################################################
Running `funannotate predict` unit testing
CMD: funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o annotate --augustus_species saccharomyces --cpus 8 --species Awesome testicus
#########################################################
-------------------------------------------------------
[May 25 02:54 PM]: OS: Debian GNU/Linux 10, 4 cores, ~ 2 GB RAM. Python: 3.8.12
[May 25 02:54 PM]: Running funannotate v1.8.10
[May 25 02:54 PM]: GeneMark not found and $GENEMARK_PATH environmental variable missing. Will skip GeneMark ab-initio prediction.
[May 25 02:54 PM]: Skipping CodingQuarry as no --rna_bam passed
[May 25 02:54 PM]: Parsed training data, run ab-initio gene predictors as follows:
  Program      Training-Method
  augustus     pretrained
  glimmerhmm   busco
  snap         busco
augustus: ERROR
    PP::Profile: Error parsing pattern file"/venv/lib/python3.8/site-packages/funannotate/config/EOG092C0B3U.prfl", line 8.
[May 25 02:54 PM]: ERROR: augustus --proteinprofile test failed, likely a compilation error. This is required to run BUSCO, exiting.
#########################################################
Traceback (most recent call last):
  File "/venv/bin/funannotate", line 8, in <module>
    sys.exit(main())
  File "/venv/lib/python3.8/site-packages/funannotate/funannotate.py", line 711, in main
    mod.main(arguments)
  File "/venv/lib/python3.8/site-packages/funannotate/test.py", line 405, in main
    runPredictTest(args)
  File "/venv/lib/python3.8/site-packages/funannotate/test.py", line 160, in runPredictTest
    assert 1500 <= countGFFgenes(os.path.join(
  File "/venv/lib/python3.8/site-packages/funannotate/test.py", line 45, in countGFFgenes
    with open(input, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'test-predict_02121c9a-e9ca-4739-9e1e-74dc80c19c83/annotate/predict_results/Awesome_testicus.gff3'

After that I went back to our cluster and ran funannotate test -t busco --cpus 8 like you suggested. This was installed via conda on our cluster. This is what happened:

funannotate test -t busco --cpus 8
#########################################################
Running `funannotate predict` BUSCO-mediated training unit testing
Downloading: https://osf.io/te2pf/download?version=1 Bytes: 1489808
CMD: funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o annotate --cpus 8 --species Awesome busco
#########################################################
-------------------------------------------------------
[May 25 03:02 PM]: OS: Ubuntu 18.04, 32 cores, ~ 66 GB RAM. Python: 3.8.12
[May 25 03:02 PM]: Running funannotate v1.8.9
[May 25 03:03 PM]: Skipping CodingQuarry as no --rna_bam passed
[May 25 03:03 PM]: Parsed training data, run ab-initio gene predictors as follows:
  Program      Training-Method
  augustus     busco
  genemark     selftraining
  glimmerhmm   busco
  snap         busco
augustus: ERROR
    PP::Profile: Error parsing pattern file"/sw/anaconda/2019.10/envs/funannotate-test/lib/python3.8/site-packages/funannotate/config/EOG092C0B3U.prfl", line 8.
[May 25 03:03 PM]: ERROR: augustus --proteinprofile test failed, likely a compilation error. This is required to run BUSCO, exiting.
#########################################################
Traceback (most recent call last):
  File "/sw/anaconda/2019.10/envs/funannotate-test/bin/funannotate", line 10, in <module>
    sys.exit(main())
  File "/sw/anaconda/2019.10/envs/funannotate-test/lib/python3.8/site-packages/funannotate/funannotate.py", line 705, in main
    mod.main(arguments)
  File "/sw/anaconda/2019.10/envs/funannotate-test/lib/python3.8/site-packages/funannotate/test.py", line 407, in main
    runBuscoTest(args)
  File "/sw/anaconda/2019.10/envs/funannotate-test/lib/python3.8/site-packages/funannotate/test.py", line 200, in runBuscoTest
    assert 1500 <= countGFFgenes(os.path.join(
  File "/sw/anaconda/2019.10/envs/funannotate-test/lib/python3.8/site-packages/funannotate/test.py", line 45, in countGFFgenes
    with open(input, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'test-busco_a5a26b86-60e5-4ee6-8d0b-8a1a3f37b3ce/annotate/predict_results/Awesome_busco.gff3'

(to make sure, I ran which augustus and it pointed to the right binary)

What other steps do you suggest me to take to help troubleshooting this?

[nextgenusfs]

Okay, so something must have changed in last build after I pushed commits yesterday, I can finally reproduce. Not sure what the problem is yet, will update here when its fixed.

(py3-funannotate) jon@Jons-MacBook-Pro:~$ docker pull nextgenusfs/funannotate
Using default tag: latest
latest: Pulling from nextgenusfs/funannotate
b03a94565ecb: Pull complete 
fdaecf38e064: Pull complete 
e1538457407a: Pull complete 
fe881bf022f5: Pull complete 
Digest: sha256:6129b60cbe393471acbe8ea49b2df8dec5e6f5a566f6e78b640b98587266a21d
Status: Downloaded newer image for nextgenusfs/funannotate:latest
docker.io/nextgenusfs/funannotate:latest
(py3-funannotate) jon@Jons-MacBook-Pro:~$ ./funannotate_dev/funannotate-docker test -t predict --cpus 4
#########################################################
Running `funannotate predict` unit testing
CMD: funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o annotate --augustus_species saccharomyces --cpus 4 --species Awesome testicus
#########################################################
-------------------------------------------------------
[May 25 10:01 AM]: OS: Debian GNU/Linux 10, 4 cores, ~ 8 GB RAM. Python: 3.8.12
[May 25 10:01 AM]: Running funannotate v1.8.10
[May 25 10:01 AM]: GeneMark not found and $GENEMARK_PATH environmental variable missing. Will skip GeneMark ab-initio prediction.
[May 25 10:01 AM]: Skipping CodingQuarry as no --rna_bam passed
[May 25 10:01 AM]: Parsed training data, run ab-initio gene predictors as follows:
  Program      Training-Method
  augustus     pretrained     
  glimmerhmm   busco          
  snap         busco          
augustus: ERROR
    PP::Profile: Error parsing pattern file"/venv/lib/python3.8/site-packages/funannotate/config/EOG092C0B3U.prfl", line 8.
[May 25 10:01 AM]: ERROR: augustus --proteinprofile test failed, likely a compilation error. This is required to run BUSCO, exiting.
#########################################################

[nextgenusfs]

Okay, the docker image is finally working again. The workaround was to use Debian apt-get to install augustus (be sure to change/update the AUGUSTUS_CONFIG_PATH from the conda version if you take this route -- you can see the Dockerfile on how I did it. So I don't know why the conda packages are failing, I tried multiple versions with the same proteinprofile error.

[Midreancgle]

How about this question？

(funannotate) ➜  [/home/yue/anaconda3/envs/funannotate/test] funannotate predict -i test.softmasked.fa --protein_evidence protein.evidence.fasta -o annotate --augustus_species saccharomyces --cpus 45 --species "Awesome testicus"
-------------------------------------------------------
[May 26 06:30 AM]: OS: Ubuntu 18.04, 56 cores, ~ 264 GB RAM. Python: 3.8.12
[May 26 06:30 AM]: Running funannotate v1.8.9
Traceback (most recent call last):
  File "/home/yue/anaconda3/envs/funannotate/bin/funannotate", line 10, in <module>
    sys.exit(main())
  File "/home/yue/anaconda3/envs/funannotate/lib/python3.8/site-packages/funannotate/funannotate.py", line 705, in main
    mod.main(arguments)
  File "/home/yue/anaconda3/envs/funannotate/lib/python3.8/site-packages/funannotate/predict.py", line 472, in main
    for f in os.listdir(os.path.join(LOCALAUGUSTUS, 'species', aug_species)):
FileNotFoundError: [Errno 2] No such file or directory: 'annotate/predict_misc/ab_initio_parameters/augustus/species/saccharomyces'

[zsdxgl]

As the log message said, the error may be caused by "compilation error". I just remove the "augustus" in conda bin. And install augustus 3.3.3 by apt. The error is solved. Good luck.

[nextgenusfs]

@Midreancgle what is the output of funannotate species, also augustus --species=help. and ls $AUGUSTUS_CONFIG_PATH. I think you might just need to re-run funannotate setup if the version of augustus changes the pre-trained parameters files can change, although the saccharomyes one I thought was stable.

[mgarl-10]

Hello. I also installed Augustus separately and it seems to be solved. Thank you very much. I run funannotate test -t predict and it goes beyond the Augustus step but now I am getting an error related with trainGlimmerHMM:

CMD ERROR: trainGlimmerHMM /ebio/test-predict_70020e58-a8e9-4c2a-9780-9a2946bfafd4/annotate/predict_misc/genome.softmasked.fa /ebio/test-predict_70020e58-a8e9-4c2a-9780-9a2946bfafd4/annotate/predict_misc/glimmer.exons -d annotate/predict_misc/glimmerhmm b"Can't locate orf.pm in @INC (you may need to install the orf module)

but it seems that all the perl modules required are installed when I run funannotate check

[caonetto]

Hello, I tried installing augustus separately, removed the conda version and updated the config path, however I am still getting the PP::Profile error. I am running Ubuntu 22. Is funannotate compatible with Augustus 3.4.0?

Thanks

[maysonlin]

Hi, I have encountered the same error. I followed up augustus installation guide on github . By the compiling step using gcc-11, I have received these 2 warning and 1 error . I am not sure if it's going to be a big issue: list.c:23:3: error: implicit declaration of function 'FatalError' is invalid in C99 [-Werror,-Wimplicit-function-declaration] FatalError ("MakeEmpty Error: Unable to allocate more memory."); ^ list.c:33:2: warning: expression result unused [-Wunused-value] exit; ^~~~ list.c:34:1: warning: non-void function does not return a value [-Wreturn-type] } ^ 2 warnings and 1 error generated. make[2]: *** [list.o] Error 1 make[1]: *** [all] Error 2 make: *** [all] Error 2

I uploaded the compiled one to HPC and used which augustus , the used cp /home/slin023/augustus/bin/augustus /home/slin023/.conda/envs/funannotate/bin/augustus to replace .conda one, but I have obtained permission issue:

Traceback (most recent call last): File "/home/slin023/.conda/envs/funannotate/bin/funannotate", line 10, in <module> sys.exit(main()) File "/home/slin023/.conda/envs/funannotate/lib/python3.8/site-packages/funannotate/funannotate.py", line 716, in main mod.main(arguments) File "/home/slin023/.conda/envs/funannotate/lib/python3.8/site-packages/funannotate/predict.py", line 594, in main augustus_version, augustus_functional = lib.checkAugustusFunc() File "/home/slin023/.conda/envs/funannotate/lib/python3.8/site-packages/funannotate/library.py", line 992, in checkAugustusFunc p1 = subprocess.Popen(['augustus', '--version'], stderr=subprocess.STDOUT, File "/home/slin023/.conda/envs/funannotate/lib/python3.8/subprocess.py", line 858, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "/home/slin023/.conda/envs/funannotate/lib/python3.8/subprocess.py", line 1704, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) PermissionError: [Errno 13] Permission denied: 'augustus' .

Even though I used chmod 755 or chmod a+x still did not work. If you have any suggestion about how to fix this , please let me know. Thank you!

[hyphaltip]

Troubleshoot by doing this outside funannotate this is external dependency so you can work on this outside the prog. which augustus

It doesn’t look like you compiled a working augustus Are you building 3.3 or 3.4?

On Sat, Jun 25, 2022 at 4:15 AM maysonlin @.***> wrote:

Hi, I have encountered the same error. I followed up augustus installation guide on github https://github.com/nextgenusfs/augustus. By the compiling step using gcc-11, I have received these 2 warning and 1 error . I am not sure if it's going to be a big issue: list.c:23:3: error: implicit declaration of function 'FatalError' is invalid in C99 [-Werror,-Wimplicit-function-declaration] FatalError ("MakeEmpty Error: Unable to allocate more memory."); ^ list.c:33:2: warning: expression result unused [-Wunused-value] exit; ^~~~ list.c:34:1: warning: non-void function does not return a value [-Wreturn-type] } ^ 2 warnings and 1 error generated. make[2]: [list.o] Error 1 make[1]: [all] Error 2 make: *** [all] Error 2

I uploaded the compiled one to HPC and used which augustus , the used cp /home/slin023/augustus/bin/augustus /home/slin023/.conda/envs/funannotate/bin/augustus to replace .conda one, but I have obtained permission issue:

Traceback (most recent call last): File "/home/slin023/.conda/envs/funannotate/bin/funannotate", line 10, in

sys.exit(main()) File "/home/slin023/.conda/envs/funannotate/lib/python3.8/site-packages/funannotate/funannotate.py", line 716, in main mod.main(arguments) File "/home/slin023/.conda/envs/funannotate/lib/python3.8/site-packages/funannotate/predict.py", line 594, in main augustus_version, augustus_functional = lib.checkAugustusFunc() File "/home/slin023/.conda/envs/funannotate/lib/python3.8/site-packages/funannotate/library.py", line 992, in checkAugustusFunc p1 = subprocess.Popen(['augustus', '--version'], stderr=subprocess.STDOUT, File "/home/slin023/.conda/envs/funannotate/lib/python3.8/subprocess.py", line 858, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "/home/slin023/.conda/envs/funannotate/lib/python3.8/subprocess.py", line 1704, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) PermissionError: [Errno 13] Permission denied: 'augustus' . Even though I used chmod 755 or chmod a+x still did not work. If you have any suggestion about how to fix this , please let me know. Thank you! — Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you commented.Message ID: ***@***.***> -- Sent from Gmail Mobile

Jason Stajich - @.***

[maysonlin]

Troubleshoot by doing this outside funannotate this is external dependency so you can work on this outside the prog. which augustus It doesn’t look like you compiled a working augustus Are you building 3.3 or 3.4? On Sat, Jun 25, 2022 at 4:15 AM maysonlin @.> wrote: Hi, I have encountered the same error. I followed up augustus installation guide on github https://github.com/nextgenusfs/augustus. By the compiling step using gcc-11, I have received these 2 warning and 1 error . I am not sure if it's going to be a big issue: list.c:23:3: error: implicit declaration of function 'FatalError' is invalid in C99 [-Werror,-Wimplicit-function-declaration] FatalError ("MakeEmpty Error: Unable to allocate more memory."); ^ list.c:33:2: warning: expression result unused [-Wunused-value] exit; ^~~~ list.c:34:1: warning: non-void function does not return a value [-Wreturn-type] } ^ 2 warnings and 1 error generated. make[2]: [list.o] Error 1 make[1]: [all] Error 2 make: [all] Error 2 I uploaded the compiled one to HPC and used which augustus , the used cp /home/slin023/augustus/bin/augustus /home/slin023/.conda/envs/funannotate/bin/augustus to replace .conda one, but I have obtained permission issue: Traceback (most recent call last): File "/home/slin023/.conda/envs/funannotate/bin/funannotate", line 10, in sys.exit(main()) File "/home/slin023/.conda/envs/funannotate/lib/python3.8/site-packages/funannotate/funannotate.py", line 716, in main mod.main(arguments) File "/home/slin023/.conda/envs/funannotate/lib/python3.8/site-packages/funannotate/predict.py", line 594, in main augustus_version, augustus_functional = lib.checkAugustusFunc() File "/home/slin023/.conda/envs/funannotate/lib/python3.8/site-packages/funannotate/library.py", line 992, in checkAugustusFunc p1 = subprocess.Popen(['augustus', '--version'], stderr=subprocess.STDOUT, File "/home/slin023/.conda/envs/funannotate/lib/python3.8/subprocess.py", line 858, in init self._execute_child(args, executable, preexec_fn, close_fds, File "/home/slin023/.conda/envs/funannotate/lib/python3.8/subprocess.py", line 1704, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) PermissionError: [Errno 13] Permission denied: 'augustus' . Even though I used chmod 755 or chmod a+x still did not work. If you have any suggestion about how to fix this , please let me know. Thank you! — Reply to this email directly, view it on GitHub <#724 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAL5O65ITD3S2PA5DNWOVTVQ25YXANCNFSM5WH5MUBA . You are receiving this because you commented.Message ID: @.> -- Sent from Gmail Mobile Jason Stajich - @.

Hi, I followed up this installation instruction to compile augustus, so it's 3.2.1 version. I compiled it at my local, then uploaded to HPC.

[nextgenusfs]

Those instructions were specific for macOS. Im not sure if the Augustus executable is static so compiling on one machine to run on another might not work. Generally this is an Augustus issue and you might get help at their GitHub repo. The apt-get version on Debian is functioning I think, at least that is what I used in docker container.

[maysonlin]

Those instructions were specific for macOS. Im not sure if the Augustus executable is static so compiling on one machine to run on another might not work. Generally this is an Augustus issue and you might get help at their GitHub repo. The apt-get version on Debian is functioning I think, at least that is what I used in docker container.

Ok, that makes sense. I will see if I can work on Debian, thank you for your reply and feedback!

[spock]

I had the same error message ("line 8" in that specific .prfl file) - when installing funannotate using mamba I got version 3.4 of augustus installed. Downgrading to the Dockerfile version specification "augustus>=3.3,<3.4" (resulting in augustus-3.3.3 | pl5321h877ab46_9 built ~6 months ago) unfortunately did not help.

The Augustus issue referenced by OnlineArts above now does have a fix applied; so hopefully the fixed version 3.4 will propagate to conda soon.

[spock]

I have worked around the problem by soft-linking a different, functional augustus 3.3.3 binary into the conda environment.

[nextgenusfs]

Yes this problem is very annoying. It's unclear what the issue with conda is, perhaps some dependency is missing I'm not sure. When I manually compile v3.4 it seems to work fine so must be something missing in the conda recipe.

[spock]

Can't locate orf.pm in @inc (you may need to install the orf module)

@mgarl-10 , did you figure out how to fix this one? I'm now having essentially the same error message:

Can't locate orf.pm in @INC (@INC contains: ~/.conda/envs/mamba/envs/funannotate/bin ~/perl5/lib/perl5/5.16.3/x86_64-linux-thread-multi ~/perl5/lib/perl5/5.16.3 ~/perl5/lib/perl5/x86_64-linux-thread-multi ~/perl5/lib/perl5 /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at ~/.conda/envs/mamba/envs/funannotate/bin/trainGlimmerHMM line 16.
BEGIN failed--compilation aborted at ~/.conda/envs/mamba/envs/funannotate/bin/trainGlimmerHMM line 16.

(I had PERL5LIB defined elsewhere in the profile, but clearing that one - and all the other irrelevant envvars - didn't solve the problem.)

orf.pm file does not exist anywhere within the environment, funannotate check says all Perl dependencies are present.

[spock]

When I manually compile v3.4 it seems to work fine so must be something missing in the conda recipe.

I've also tried the new/fixed v3.4 first, but it kept asking for commandline parameters json file, which is why I've reverted to 3.3.3 binary.

[spock]

orf.pm file does not exist anywhere within the environment, funannotate check says all Perl dependencies are present.

alright, I've mamba-installed funannotate 1.8.11 (the previous available version, 1.8.12 was not available), and found orf.pm in that version 😀

~/.conda/envs/mamba/envs/funannotate1_8_11/share/glimmerhmm/train/orf.pm

The entire share/glimmerhmm directory is absent from 1.8.13 🤯 - so I've copied it in. I also had to replace the bin/trainGlimmerHMM script, because it was also broken in several ways 🤦🏻‍♂️

This did solve the problem, funannotate test -t predict now completes successfully!

My previous perception "conda is a nice and easy way to install pkgs with lots of dependencies" has now degraded a lot. Next time I decide to upgrade funannotate I will probably go the "build singularity from docker" route 😄

[nextgenusfs]

So once you have a working environment you should mostly be able to just upgrade funannotate with pip in that environment. I have love/hate view if conda as well....

[tjhinet]

Hi nextgenusfs and Spock, I am trying to solve the same error message ("line 8" in the .prfl file). I have tried removing the augustus from my funannotate conda environment and soft-linking v3.4 augustus at first. As you have mentioned, Spock, this did not work. May I ask which version 3.3.3 of augustus worked for you? Thank you for the help!

[spock]

May I ask which version 3.3.3 of augustus worked for you?

@tjhinet , while my OS-built augustus 3.3.3 did pass test -t predict, it failed later during the actual run - because of an lp_solve library incompatibility. (Now that I write this - I could have probably modified LD_LIBRARY_PATH.... but that's likely to break something else.) I have also tried force-installing other/older builds of augustus into the conda env, but none of them passed the test - or there were env conflicts that conda could not resolve (from build augustus=3.3.3=pl5262h5a9fe7b_6 and older).

Looking at the augustus issue, this is caused by a change in standard libraries; so essentially one needs either "older standard libs with before-fix augustus" or "current/newer standard libs with after-the-fix augustus 3.4". One could probably still construct a conda environment that would work (with e.g. augustus=3.3.3=pl5262h5a9fe7b_6, which might be functional), but after so much trial and error I wanted an orthogonal fix 😆

So I've built a singularity image (from docker image) yesterday, and it appears to have passed test -t all without any error/warning messages. So I'll give it a try today. Very much hope this will finally work, as I now have several genomes stuck because of the broken predict stage.

[nextgenusfs]

In the docker build I just used the version in apt-get which is functional, so if you are on Debian that is the quickest solution. On my Mac I was able to build the latest source code from Augustus repo and that also worked.

[spock]

So I've built a singularity image (from docker image) yesterday, and it appears to have passed test -t all without any error/warning messages. So I'll give it a try today. Very much hope this will finally work, as I now have several genomes stuck because of the broken predict stage.

Everything went through without hiccups of any kind.

Singularity can be built in one of these two ways, I've used the second one:

# option 1
singularity build funannotate.sif docker://nextgenusfs/funannotate
# option 2
docker run -v /var/run/docker.sock:/var/run/docker.sock -v /abs_path_to_your_cwd:/output --privileged -t --rm quay.io/singularity/docker2singularity --name nextgenusfs_funannotate nextgenusfs/funannotate

[spock]

On my Mac I was able to build the latest source code from Augustus repo and that also worked.

how do you get around that command-line-parameters.json requirement with Augustus 3.4.0? I did copy in the file, but then it also wanted one in the project folder; that's where I've stopped.

[bernt-matthias]

Seems that the problem has been solved upstream https://github.com/Gaius-Augustus/Augustus/issues/346 .. but there was no new release. Just trying to patch this into the augustus bioconda recipe https://github.com/bioconda/bioconda-recipes/pull/36871

[nextgenusfs]

Great. Let me know if you need a review for bioconda.

[myco-bradshaw]

Seems that the problem has been solved upstream Gaius-Augustus/Augustus#346 .. but there was no new release. Just trying to patch this into the augustus bioconda recipe bioconda/bioconda-recipes#36871

Hello bernt-matthias, is there a current workaround for installing funannotate through bioconda with this patch? Or should I install a previous version (on linux system). I ask because mamba create -n funannotate funannotate command runs into the same orf issue during the BUSCO run on a clean install.

[bernt-matthias]

Unfortunately not, I tried in https://github.com/bioconda/bioconda-recipes/pull/36870 .. the test over there are successful. But the tool actually still fails (when tested more thoroughly).

The reason seems to be that bioconda (for some unknown reason) still uses outdated requirements.

Have no solution so far.

[myco-bradshaw]

@bernt-matthias It looks like Augustus has updated their version from 3.4 to 3.5 on their GitHub page. Hopefully, a conda package update is soon to follow.

[hyphaltip]

For this problem you need to check what the glimmerhmm package is that is installed: you need to make sure it is build >=3 --> works

conda list glimmerhmm
# packages in environment at XXX:
#
# Name                    Version                   Build  Channel
glimmerhmm                3.0.4           pl526hf484d3e_3    bioconda

Does not work:

conda list glimmerhmm
# packages in environment at XXX 
#
# Name                    Version                   Build  Channel
glimmerhmm                3.0.4                h2d50403_2    bioconda

[caonetto]

Has anyone been able to make the conda install work? Ive tried basically all the options metioned above, including the install of augustus independently and removing the augustus executable from the bin conda environment folder but I am still getting the same errors.