maxulysse
commented
4 years ago Love the idea
Open ewels opened 4 years ago
maxulysse
commented
4 years ago Love the idea
ewels
commented
4 years ago Having looked into this more, I think that we are best off keeping our JSON Schema. But we should be able to build a Bioschema fairly easy. Where exactly this should go or where it would be used is less clear.
ewels
commented
4 years ago For examples, can look at the HTML source of any workflow on https://workflowhub.eu.
For example: https://workflowhub.eu/workflows/38
ewels
commented
4 years ago Another example: https://workflowhub.eu/workflows/19 (for nf-core/viralrecon)
![\"nf-core/viralrecon\"](\"https://raw.githubusercontent.com/nf-core/viralrecon/master/docs/images/nf-core-viralrecon_logo.png\"){kind=logo}
\n![\"GitHub](\"https://github.com/nf-core/viralrecon/workflows/nf-core%20CI/badge.svg\"){kind=icon}
![\"GitHub](\"https://github.com/nf-core/viralrecon/workflows/nf-core%20linting/badge.svg\"){kind=icon}
![\"Nextflow\"](\"https://img.shields.io/badge/nextflow-%E2%89%A519.10.0-brightgreen.svg\"){kind=icon}
![\"install](\"https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg\"){kind=icon}
![\"Docker\"](\"https://img.shields.io/docker/automated/nfcore/viralrecon.svg\")
![\"DOI\"](\"https://zenodo.org/badge/DOI/10.5281/zenodo.3872730.svg\"){kind=icon}
nfcore/viralrecon is a bioinformatics analysis pipeline used to perform assembly and intrahost/low-frequency variant calling for viral samples. The pipeline currently supports metagenomics and amplicon sequencing data derived from the Illumina sequencing platform.
\r\nThis pipeline is a re-implementation of the SARS_Cov2_consensus-nf and SARS_Cov2_assembly-nf pipelines initially developed by Sarai Varona and Sara Monzon from BU-ISCIII. Porting both of these pipelines to nf-core was an international collaboration between numerous contributors and developers, led by Harshil Patel from the The Bioinformatics & Biostatistics Group at The Francis Crick Institute, London. We appreciated the need to have a portable, reproducible and scalable pipeline for the analysis of COVID-19 sequencing samples and so the Avengers Assembled! Please come and join us and add yourself to the contributor list :)
\r\nWe have integrated a number of options in the pipeline to allow you to run specific aspects of the workflow if you so wish. For example, you can skip all of the assembly steps with the --skip_assembly parameter. See usage docs for all of the available options when running the pipeline.
Please click here to see an example MultiQC report generated using the parameters defined in this configuration file to run the pipeline on samples which were prepared from the ncov-2019 ARTIC Network V1 amplicon set and sequenced on the Illumina MiSeq platform in 301bp paired-end format.
\r\nThe pipeline is built using Nextflow, a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It comes with docker containers making installation trivial and results highly reproducible. Furthermore, automated continuous integration tests to run the pipeline on a full-sized dataset are passing on AWS cloud.
\r\nENA FTP, parallel-fastq-dump; if required)cat; if required)FastQC)fastp)Bowtie 2)SAMtools)iVar; amplicon data only)picard; removal optional)picard, SAMtools)VarScan 2, BCFTools, BEDTools || iVar variants and consensus || BCFTools, BEDTools)SnpEff, SnpSift)QUAST)Cutadapt; amplicon data only)Kraken 2)SPAdes || metaSPAdes || Unicycler || minia)blastn)ABACAS)PlasmidID)QUAST)Minimap2, seqwish, vg, Bandage)SnpEff, SnpSift)MultiQC)i. Install nextflow
ii. Install either Docker or Singularity for full pipeline reproducibility (please only use Conda as a last resort; see docs)
iii. Download the pipeline and test it on a minimal dataset with a single command
\r\nnextflow run nf-core/viralrecon -profile test,<docker/singularity/conda/institute>\r\n\r\n\r\nPlease check nf-core/configs to see if a custom config file to run nf-core pipelines already exists for your Institute. If so, you can simply use
\r\n-profile <institute>in your command. This will enable eitherdockerorsingularityand set the appropriate execution settings for your local compute environment.
iv. Start running your own analysis!
\r\nnextflow run nf-core/viralrecon -profile <docker/singularity/conda/institute> --input samplesheet.csv --genome 'NC_045512.2' -profile docker\r\nSee usage docs for all of the available options when running the pipeline.
\r\nThe nf-core/viralrecon pipeline comes with documentation about the pipeline, found in the docs/ directory:
These scripts were originally written by Sarai Varona, Miguel Juliá and Sara Monzon from BU-ISCIII and co-ordinated by Isabel Cuesta for the Institute of Health Carlos III, Spain. Through collaboration with the nf-core community the pipeline has now been updated substantially to include additional processing steps, to standardise inputs/outputs and to improve pipeline reporting; implemented primarily by Harshil Patel from The Bioinformatics & Biostatistics Group at The Francis Crick Institute, London.
\r\nMany thanks to others who have helped out and contributed along the way too, including (but not limited to):
\r\n\r\n\r\n\r\nName\r\nAffiliation\r\n\r\n\r\n\r\n\r\nAlexander Peltzer\r\nBoehringer Ingelheim, Germany\r\n\r\n\r\nAlison Meynert\r\nUniversity of Edinburgh, Scotland\r\n\r\n\r\nEdgar Garriga Nogales\r\nCentre for Genomic Regulation, Spain\r\n\r\n\r\nErik Garrison\r\nUCSC, USA\r\n\r\n\r\nGisela Gabernet\r\nQBiC, University of Tübingen, Germany\r\n\r\n\r\nJoao Curado\r\nFlomics Biotech, Spain\r\n\r\n\r\nJose Espinosa-Carrasco\r\nCentre for Genomic Regulation, Spain\r\n\r\n\r\nKatrin Sameith\r\nDRESDEN-concept Genome Center, Germany\r\n\r\n\r\nLluc Cabus\r\nFlomics Biotech, Spain\r\n\r\n\r\nMarta Pozuelo\r\nFlomics Biotech, Spain\r\n\r\n\r\nMaxime Garcia\r\nSciLifeLab, Sweden\r\n\r\n\r\nMichael Heuer\r\nUC Berkeley, USA\r\n\r\n\r\nPhil Ewels\r\nSciLifeLab, Sweden\r\n\r\n\r\nSimon Heumos\r\nQBiC, University of Tübingen, Germany\r\n\r\n\r\nStephen Kelly\r\nMemorial Sloan Kettering Cancer Center, USA\r\n\r\n\r\nThanh Le Viet\r\nQuadram Institute, UK\r\n\r\n\r\n\r\n\r\n\r\nListed in alphabetical order
\r\n
If you would like to contribute to this pipeline, please see the contributing guidelines.
\r\nFor further information or help, don’t hesitate to get in touch on Slack (you can join with this invite).
\r\nIf you use nf-core/viralrecon for your analysis, please cite it using the following doi: 10.5281/zenodo.3872730
\r\nAn extensive list of references for the tools used by the pipeline can be found in the CITATIONS.md file.
You can cite the nf-core publication as follows:
\r\n\r\n", "name": "nf-core/viralrecon", "url": "https://workflowhub.eu/workflows/19", "keywords": "covid-19", "license": "https://opensource.org/licenses/MIT", "creator": [ { "@type": "Person", "name": "Sarai Varona and Miguel Juliá and Sara Monzon and Alexander Peltzer and Alison Meynert and Edgar Garriga Nogales and Erik Garrison and Gisela Gabernet and Harshil Patel and Joao Curado and Jose Espinosa-Carrasco and Katrin Sameith and Marta Pozuelo and Maxime Garcia and Michael Heuer and Phil Ewels and Simon Heumos and Stephen Kelly and Thanh Le Viet and Isabel Cuesta" } ], "producer": [ { "@type": [ "Project", "Organization" ], "@id": "https://workflowhub.eu/projects/5", "name": "nf-core viralrecon" } ], "dateCreated": "2020-05-14 14:10:58 UTC", "dateModified": "2020-06-02 10:46:31 UTC", "encodingFormat": "application/zip", "sdPublisher": [ { "@type": "Person", "@id": "https://workflowhub.eu/people/11", "name": "Hervé Ménager" } ], "version": 1, "programmingLanguage": "Nextflow", "inputs": [ ], "outputs": [ ] } ```The nf-core framework for community-curated bioinformatics pipelines.
\r\nPhilip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.
\r\nNat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.
\r\n
\r\nReadCube: Full Access Link
ewels
commented
3 years ago I think that this would be a great file to generate automatically to give pipeline-level metadata of all things possible in a flat file.
Can hopefully generate a lot of it automatically from nextflow config results, plus stuff from the schema.
ewels
commented
3 years ago I'm not super clear of the overlap between Bioschemas + RO Crate (https://github.com/nf-core/tools/pull/703) so need to think about that when working on either 👍🏻
We are close to releasing our new pipeline schema which describe input parameters. Through chatting to people at BOSC 🍐 I've become aware of a project called Bioschemas which works with schema.org to publish official standardised ways to describe things.
They are currently developing a new schema standard which is specifically for computational workflows: https://bioschemas.org/profiles/Workflow/
Aspects of this schema seem to share a lot of overlap with the schema structure that we're working with already. It could be interesting to see if we can migrate our schema structure to the Bioschemas structure to adhere to this new community standard. This would help the visibility of the nf-core pipelines across the web and in other projects (eg. WorkflowHub).
This will be conditional on it being relatively fast and painless to implement and it still providing all of the functionality that we need (eg. custom fields, grouping of input parameters and so on).