search

NBISweden / pipelines-nextflow

A set of workflows written in Nextflow for Genome Annotation.
GNU General Public License v3.0
42 stars 18 forks source link

agat version updated to 0.4.0, agat_sp_split_by_level2_feature.pl rep… #43

Closed royfrancis closed 4 years ago

royfrancis commented 4 years ago

agat version updated to 0.4.0, agat_sp_split_by_level2_feature.pl replaced with agat_sp_feature_by_record_type.pl

mahesh-panchal commented 4 years ago

Can you include the results of testing with the test profiles too please?

nextflow run -profile nbis,{singularity,conda},test [ -c <config> ] <workflow>
royfrancis commented 4 years ago

AbinitioTraining and FunctionalAnnotation both failed using Singularity but completed without errors using conda environment. Run on NAC.

Logs below:

AbinitioTraining Singularity ``` nextflow run -profile nbis,singularity,test ../pipelines-nextflow/AbinitioTraining/AbinitioTraining.nf N E X T F L O W ~ version 20.01.0 Launching `../pipelines-nextflow/AbinitioTraining/AbinitioTraining.nf` [mad_khorana] - revision: da0986b55d WARN: It appears you have never run this project before -- Option `-resume` is ignored WARN: DSL 2 IS AN EXPERIMENTAL FEATURE UNDER DEVELOPMENT -- SYNTAX MAY CHANGE IN FUTURE RELEASE NBIS _ _ ____ _____ _____ | \ | | _ \_ _|/ ____| | \| | |_) || | | (___ | . ` | _ < | | \___ \ | |\ | |_) || |_ ____) | |_| \_|____/_____|_____/ Annotation Service Abintio training dataset workflow =================================== General Parameters maker_evidence_gff : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genes.gff genome : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genome.fa outdir : results species_label : test_species Model selection by AED model_selection_value : 0.3 Filter by locus distance locus_distance : 500 Protein Sequence extraction parameters codon_table : 1 Augustus training parameters test_size : 10 flank_region_size : 500 [- ] process > abinitio_training:split_maker_evidence - [- ] process > abinitio_training:model_selection_by_AED - [- ] process > abinitio_training:retain_longest_isoform - [- ] process > abinitio_training:remove_incomplete_gene_models - [- ] process > abinitio_training:filter_by_locus_distance - [- ] process > abinitio_training:extract_protein_sequence - [- ] process > abinitio_training:blast_makeblastdb - [- ] process > abinitio_training:blast_recursive - [- ] process > abinitio_training:gff_filter_by_blast - [- ] process > abinitio_training:split_maker_evidence [ 0%] 0 of 1 [- ] process > abinitio_training:model_selection_by_AED - [- ] process > abinitio_training:retain_longest_isoform - [- ] process > abinitio_training:remove_incomplete_gene_models - [- ] process > abinitio_training:filter_by_locus_distance - [- ] process > abinitio_training:extract_protein_sequence - ... executor > slurm (10) [85/c0879d] process > abinitio_training:split_maker_evidence [100%] 1 of 1 ✔ [8f/6b0d07] process > abinitio_training:model_selection_by_AED [100%] 1 of 1 ✔ [93/0ad326] process > abinitio_training:retain_longest_isoform [100%] 1 of 1 ✔ [bb/33ba3b] process > abinitio_training:remove_incomplete_gene_models [100%] 1 of 1 ✔ [26/b5d499] process > abinitio_training:filter_by_locus_distance [100%] 1 of 1 ✔ [b1/5befb9] process > abinitio_training:extract_protein_sequence [100%] 1 of 1 ✔ [8d/9b3486] process > abinitio_training:blast_makeblastdb [100%] 1 of 1 ✔ [9e/52b9b1] process > abinitio_training:blast_recursive [100%] 1 of 1 ✔ [09/956f29] process > abinitio_training:gff_filter_by_blast [100%] 1 of 1 ✔ [- ] process > abinitio_training:gff2gbk - [- ] process > abinitio_training:gbk2augustus - [- ] process > abinitio_training:augustus_training - [bd/d8d258] process > abinitio_training:convert_gff2zff [100%] 1 of 1 ✔ [- ] process > abinitio_training:snap_training - Pulling Singularity image docker://quay.io/biocontainers/snap:2013_11_29--h516909a_2 [cache /home/roy/test/at/work/singularity/quay.io-biocontainers-snap-2013_11_29--h516909a_2.img] Error executing process > 'abinitio_training:gff2gbk (codingGeneFeatures.filter.longest_cds.complete.good_distance_blast-filtered)' Caused by: Failed to pull singularity image command: singularity pull --name quay.io-biocontainers-augustus-3.3.3--pl526hce533f5_1.img docker://quay.io/biocontainers/augustus:3.3.3--pl526hce533f5_1 > /dev/null status : 255 message: [34mINFO: Converting OCI blobs to SIF format INFO: Starting build... Getting image source signatures Copying blob sha256:1dbcab28ce46b65c0174e5e82658492107396fead31e9144c343e6bc96e471c7 Copying blob sha256:cfb1ba34637d96787f6e780192bc5f09c04fe0b40c4e1acdcbc88953bce25b5b Copying blob sha256:ace2d8a63dd59063206661355f22da9e1c75c56d6cae5116d91db7a65f640667 Copying blob sha256:75c080ef15ebb28daeadc91eeda0ca95a893fe1f4e55845da3feb58069d42fdc Copying blob sha256:316957f8baafcefd4c9f72b7e3f3fafb7fcb3ca805dc69d242aef656e0736490 Copying blob sha256:dbd31e1d863d4a41565c08ad1d419c3a3538abfdd4a7971de63b8c5c8a43caec Copying blob sha256:2f8531d5a6ece34772cf99a2d30e3057c636f1ad05cdec0c8009eab401e616a7 Copying blob sha256:1dbcab28ce46b65c0174e5e82658492107396fead31e9144c343e6bc96e471c7 Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 executor > slurm (10) [85/c0879d] process > abinitio_training:split_maker_evidence [100%] 1 of 1 ✔ [8f/6b0d07] process > abinitio_training:model_selection_by_AED [100%] 1 of 1 ✔ [93/0ad326] process > abinitio_training:retain_longest_isoform [100%] 1 of 1 ✔ [bb/33ba3b] process > abinitio_training:remove_incomplete_gene_models [100%] 1 of 1 ✔ [26/b5d499] process > abinitio_training:filter_by_locus_distance [100%] 1 of 1 ✔ [b1/5befb9] process > abinitio_training:extract_protein_sequence [100%] 1 of 1 ✔ [8d/9b3486] process > abinitio_training:blast_makeblastdb [100%] 1 of 1 ✔ [9e/52b9b1] process > abinitio_training:blast_recursive [100%] 1 of 1 ✔ [09/956f29] process > abinitio_training:gff_filter_by_blast [100%] 1 of 1 ✔ [- ] process > abinitio_training:gff2gbk - [- ] process > abinitio_training:gbk2augustus - [- ] process > abinitio_training:augustus_training - [bd/d8d258] process > abinitio_training:convert_gff2zff [100%] 1 of 1 ✔ [- ] process > abinitio_training:snap_training - Pulling Singularity image docker://quay.io/biocontainers/snap:2013_11_29--h516909a_2 [cache /home/roy/test/at/work/singularity/quay.io-biocontainers-snap-2013_11_29--h516909a_2.img] Oops .. something went wrong Error executing process > 'abinitio_training:gff2gbk (codingGeneFeatures.filter.longest_cds.complete.good_distance_blast-filtered)' Caused by: Failed to pull singularity image command: singularity pull --name quay.io-biocontainers-augustus-3.3.3--pl526hce533f5_1.img docker://quay.io/biocontainers/augustus:3.3.3--pl526hce533f5_1 > /dev/null status : 255 message: [34mINFO: Converting OCI blobs to SIF format INFO: Starting build... Getting image source signatures Copying blob sha256:1dbcab28ce46b65c0174e5e82658492107396fead31e9144c343e6bc96e471c7 Copying blob sha256:cfb1ba34637d96787f6e780192bc5f09c04fe0b40c4e1acdcbc88953bce25b5b Copying blob sha256:ace2d8a63dd59063206661355f22da9e1c75c56d6cae5116d91db7a65f640667 Copying blob sha256:75c080ef15ebb28daeadc91eeda0ca95a893fe1f4e55845da3feb58069d42fdc Copying blob sha256:316957f8baafcefd4c9f72b7e3f3fafb7fcb3ca805dc69d242aef656e0736490 Copying blob sha256:dbd31e1d863d4a41565c08ad1d419c3a3538abfdd4a7971de63b8c5c8a43caec Copying blob sha256:2f8531d5a6ece34772cf99a2d30e3057c636f1ad05cdec0c8009eab401e616a7 Copying blob sha256:1dbcab28ce46b65c0174e5e82658492107396fead31e9144c343e6bc96e471c7 Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 Copying blob sha256:2e178fd72baf28bc587b5d274574f1a3d3e25ac802fdaf154af258a3c5891831 executor > slurm (10) [85/c0879d] process > abinitio_training:split_maker_evidence [100%] 1 of 1 ✔ [8f/6b0d07] process > abinitio_training:model_selection_by_AED [100%] 1 of 1 ✔ [93/0ad326] process > abinitio_training:retain_longest_isoform [100%] 1 of 1 ✔ [bb/33ba3b] process > abinitio_training:remove_incomplete_gene_models [100%] 1 of 1 ✔ [26/b5d499] process > abinitio_training:filter_by_locus_distance [100%] 1 of 1 ✔ [b1/5befb9] process > abinitio_training:extract_protein_sequence [100%] 1 of 1 ✔ [8d/9b3486] process > abinitio_training:blast_makeblastdb [100%] 1 of 1 ✔ [9e/52b9b1] process > abinitio_training:blast_recursive [100%] 1 of 1 ✔ [09/956f29] process > abinitio_training:gff_filter_by_blast [100%] 1 of 1 ✔ [- ] process > abinitio_training:gff2gbk - [- ] process > abinitio_training:gbk2augustus - [- ] process > abinitio_training:augustus_training - [bd/d8d258] process > abinitio_training:convert_gff2zff [100%] 1 of 1 ✔ [- ] process > abinitio_training:snap_training - Pulling Singularity image docker://quay.io/biocontainers/snap:2013_11_29--h516909a_2 [cache /home/roy/test/at/work/singularity/quay.io-biocontainers-snap-2013_11_29--h516909a_2.img] Oops .. something went wrong Error executing process > 'abinitio_training:gff2gbk (codingGeneFeatures.filter.longest_cds.complete.good_distance_blast-filtered)' Caused by: Failed to pull singularity image command: singularity pull --name quay.io-biocontainers-augustus-3.3.3--pl526hce533f5_1.img docker://quay.io/biocontainers/augustus:3.3.3--pl526hce533f5_1 > /dev/null status : 255 message: [34mINFO: Converting OCI blobs to SIF format INFO: Starting build... Getting image source signatures Copying blob sha256:1dbcab28ce46b65c0174e5e82658492107396fead31e9144c343e6bc96e471c7 Copying blob sha256:cfb1ba34637d96787f6e780192bc5f09c04fe0b40c4e1acdcbc88953bce25b5b Copying blob sha256:ace2d8a63dd59063206661355f22da9e1c75c56d6cae5116d91db7a65f640667 Copying blob sha256:75c080ef15ebb28daeadc91eeda0ca95a893fe1f4e55845da3feb58069d42fdc Copying blob sha256:316957f8baafcefd4c9f72b7e3f3fafb7fcb3ca805dc69d242aef656e0736490 Copying blob sha256:dbd31e1d863d4a41565c08ad1d419c3a3538abfdd4a7971de63b8c5c8a43caec Copying blob sha256:2f8531d5a6ece34772cf99a2d30e3057c636f1ad05cdec0c8009eab401e616a7 Copying blob sha256:1dbcab28ce46b65c0174e5e82658492107396fead31e9144c343e6bc96e471c7 Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 Copying blob sha256:2e178fd72baf28bc587b5d274574f1a3d3e25ac802fdaf154af258a3c5891831 Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 Copying blob sha256:7fe17215b108da0e73b3e595507e2a8bca68c7b4f262ec41aaf19ccd9ce91609 Copying config sha256:e67554a76257741979b43595fb1dc926fd3cd091941ec7c4f84fbe804a47ff04 Writing manifest to image destination Storing signatures FATAL: While making image from oci registry: while building SIF from layers: conveyor failed to get: no descriptor found for reference "a0dd34a79b4a962be8339533e850194c7621ff333132a974d9adf4026a22c176" ```
AbinitioTraining Conda ``` nextflow run -profile nbis,conda,test ../pipelines-nextflow/AbinitioTraining/AbinitioTraining.nf N E X T F L O W ~ version 20.01.0 Launching `../pipelines-nextflow/AbinitioTraining/AbinitioTraining.nf` [naughty_stallman] - revision: da0986b55d WARN: DSL 2 IS AN EXPERIMENTAL FEATURE UNDER DEVELOPMENT -- SYNTAX MAY CHANGE IN FUTURE RELEASE NBIS _ _ ____ _____ _____ | \ | | _ \_ _|/ ____| | \| | |_) || | | (___ | . ` | _ < | | \___ \ | |\ | |_) || |_ ____) | |_| \_|____/_____|_____/ Annotation Service Abintio training dataset workflow =================================== General Parameters maker_evidence_gff : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genes.gff genome : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genome.fa outdir : results species_label : test_species Model selection by AED model_selection_value : 0.3 Filter by locus distance locus_distance : 500 Protein Sequence extraction parameters codon_table : 1 Augustus training parameters test_size : 10 flank_region_size : 500 [- ] process > abinitio_training:split_maker_evidence - [- ] process > abinitio_training:model_selection_by_AED - [- ] process > abinitio_training:retain_longest_isoform - [- ] process > abinitio_training:remove_incomplete_gene_models - [- ] process > abinitio_training:split_maker_evidence - [- ] process > abinitio_training:model_selection_by_AED - [- ] process > abinitio_training:retain_longest_isoform - [- ] process > abinitio_training:remove_incomplete_gene_models - [- ] process > abinitio_training:filter_by_locus_distance - [- ] process > abinitio_training:extract_protein_sequence - [- ] process > abinitio_training:blast_makeblastdb - [- ] process > abinitio_training:blast_recursive - [- ] process > abinitio_training:gff_filter_by_blast - [- ] process > abinitio_training:gff2gbk - [- ] process > abinitio_training:gbk2augustus - [- ] process > abinitio_training:split_maker_evidence - [- ] process > abinitio_training:model_selection_by_AED - [- ] process > abinitio_training:retain_longest_isoform - [- ] process > abinitio_training:remove_incomplete_gene_models - [- ] process > abinitio_training:filter_by_locus_distance - [- ] process > abinitio_training:extract_protein_sequence - [- ] process > abinitio_training:blast_makeblastdb - [- ] process > abinitio_training:blast_recursive - executor > slurm (1) ... ... executor > slurm (14) [1a/b47d3e] process > abinitio_training:split_maker_evidence [100%] 1 of 1 ✔ [a1/6b9bd6] process > abinitio_training:model_selection_by_AED [100%] 1 of 1 ✔ [98/957e9c] process > abinitio_training:retain_longest_isoform [100%] 1 of 1 ✔ [d7/bed60c] process > abinitio_training:remove_incomplete_gene_models [100%] 1 of 1 ✔ [9a/7efe65] process > abinitio_training:filter_by_locus_distance [100%] 1 of 1 ✔ [c3/2410f2] process > abinitio_training:extract_protein_sequence [100%] 1 of 1 ✔ [d0/9cc94c] process > abinitio_training:blast_makeblastdb [100%] 1 of 1 ✔ [58/0f7e38] process > abinitio_training:blast_recursive [100%] 1 of 1 ✔ [b3/e8e8b2] process > abinitio_training:gff_filter_by_blast [100%] 1 of 1 ✔ [10/37297c] process > abinitio_training:gff2gbk [100%] 1 of 1 ✔ [35/c3f735] process > abinitio_training:gbk2augustus [100%] 1 of 1 ✔ [fa/6ec457] process > abinitio_training:augustus_training [100%] 1 of 1 ✔ [a3/c82181] process > abinitio_training:convert_gff2zff [100%] 1 of 1 ✔ [d2/17f5fc] process > abinitio_training:snap_training [100%] 1 of 1 ✔ Augustus training dataset complete! Completed at: 24-Jun-2020 12:06:24 Duration : 15m 22s CPU hours : (a few seconds) Succeeded : 14 ```
FunctionalAnnotation Singularity ``` nextflow run -profile nbis,singularity,test ../pipelines-nextflow/FunctionalAnnotation/FunctionalAnnotation.nf N E X T F L O W ~ version 20.01.0 Launching `../pipelines-nextflow/FunctionalAnnotation/FunctionalAnnotation.nf` [tender_dalembert] - revision: 6c75d797eb WARN: DSL 2 IS AN EXPERIMENTAL FEATURE UNDER DEVELOPMENT -- SYNTAX MAY CHANGE IN FUTURE RELEASE NBIS _ _ ____ _____ _____ | \ | | _ \_ _|/ ____| | \| | |_) || | | (___ | . ` | _ < | | \___ \ | |\ | |_) || |_ ____) | |_| \_|____/_____|_____/ Annotation Service Functional annotation workflow =================================== General parameters gff_annotation : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genes.gff genome : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genome.fa outdir : results Parallelisation parameters records_per_file : 1000 Gff2Protein parameters codon_table : 1 Blast parameters blast_db_fasta : /projects/references/databases/uniprot/2018-03/uniprot_sprot.fasta blast_evalue : 1e-6 Interproscan parameters interproscan_db : Merge functional annotation parameters id_prefix : NBIS protein_existence : 5 executor > slurm (1) [fe/f15e73] process > functional_annotation:gff2protein [100%] 1 of 1, cached: 1 ✔ executor > slurm (1) [fe/f15e73] process > functional_annotation:gff2protein [100%] 1 of 1, cached: 1 ✔ [- ] process > functional_annotation:blastp - [- ] process > functional_annotation:interproscan - executor > slurm (1) [fe/f15e73] process > functional_annotation:gff2protein [100%] 1 of 1, cached: 1 ✔ [- ] process > functional_annotation:blastp - [- ] process > functional_annotation:interproscan - [- ] process > functional_annotation:merge_functional_annotation - Pulling Singularity image docker://biocontainers/interproscan [cache /home/roy/test/fa/work/singularity/biocontainers-interproscan.img] Oops .. something went wrong WARN: Singularity cache directory has not been defined -- Remote image will be stored in the path: /home/roy/test/fa/work/singularity WARN: Killing pending tasks (1) Error executing process > 'functional_annotation:interproscan (1)' Caused by: Failed to pull singularity image command: singularity pull --name biocontainers-interproscan.img docker://biocontainers/interproscan > /dev/null status : 255 message: [31mFATAL: While making image from oci registry: failed to get checksum for docker://biocontainers/interproscan: Error reading manifest latest in docker. io/biocontainers/interproscan: manifest unknown: manifest unknown ```
FunctionalAnnotation Conda ``` nextflow run -profile nbis,conda,test ../pipelines-nextflow/FunctionalAnnotation/FunctionalAnnotation.nf N E X T F L O W ~ version 20.01.0 Launching `../pipelines-nextflow/FunctionalAnnotation/FunctionalAnnotation.nf` [evil_curran] - revision: 6c75d797eb WARN: DSL 2 IS AN EXPERIMENTAL FEATURE UNDER DEVELOPMENT -- SYNTAX MAY CHANGE IN FUTURE RELEASE NBIS _ _ ____ _____ _____ | \ | | _ \_ _|/ ____| | \| | |_) || | | (___ | . ` | _ < | | \___ \ | |\ | |_) || |_ ____) | |_| \_|____/_____|_____/ Annotation Service Functional annotation workflow =================================== General parameters gff_annotation : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genes.gff genome : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genome.fa outdir : results Parallelisation parameters records_per_file : 1000 Gff2Protein parameters codon_table : 1 Blast parameters blast_db_fasta : /projects/references/databases/uniprot/2018-03/uniprot_sprot.fasta blast_evalue : 1e-6 Interproscan parameters interproscan_db : Merge functional annotation parameters id_prefix : NBIS protein_existence : 5 [- ] process > functional_annotation:gff2protein - executor > slurm (1) [d6/288dc9] process > functional_annotation:gff2protein [100%] 1 of 1 ✔ executor > slurm (4) [d6/288dc9] process > functional_annotation:gff2protein [100%] 1 of 1 ✔ [39/91f7da] process > functional_annotation:blastp [100%] 1 of 1 ✔ [1e/545b13] process > functional_annotation:interproscan [100%] 1 of 1 ✔ [5b/366c2e] process > functional_annotation:merge_functional_annotation [100%] 1 of 1 ✔ Functional annotation input preparation complete! Completed at: 24-Jun-2020 12:18:16 Duration : 27m 52s CPU hours : 2.9 Succeeded : 4 ```
mahesh-panchal commented 4 years ago

Fixes #42