ndreey
commented
6 months ago Setting everything up
As with bioinformatics, programs are not ready to go from the get go.
UPPMAX does not update the nf-core module anymore. Hence, the nf-core/mag pipeline is 2.3.2 while the latest is 3.0.0 and the nf-core tools is 2.6 while latest is 2.14.1. Furthermore, Nextflow is also outdated as i got this error Nextflow version 22.10.2 does not match workflow required version: >=23.04.0. Noticed this when i tried running the test profile and i ran into errors that were fixed in the later versions.
To fix this, i created a nf-core mamba environment and installed Nextflow and nf-core.
mamba create -n nf-core python=3.12 nf-core nextflowNow, with the .bashrc edits (#37) i activated the environment and ran the test profile together with the uppmax profile; SUCCESS!.
mamba activate nf-core
mkdir nf-core-mag-test
cd nf-core-mag-test
# Run the workflow
nextflow run nf-core/mag -profile test,uppmax --project naiss2024-5-1 --outdir .It is running now and creating slurm jobs.
Errors from UPPMAX modules
This is the code i rand to get the errors mentioned above.
module load nf-core/latest nf-core-pipelines/latest
# Path to were all pipelines are stored
echo $NF_CORE_PIPELINES
/sw/bioinfo/nf-core-pipelines/latest/rackham
tree -L 2 $NF_CORE_PIPELINES -I 'singularity_cache_dir'
├── mag
│ ├── 1.0.0
│ ├── 1.1.0
│ ├── 1.1.1
│ ├── 1.1.2
│ ├── 1.2.0
│ ├── 2.0.0
│ ├── 2.1.0
│ ├── 2.1.1
│ ├── 2.2.0
│ ├── 2.2.1
│ ├── 2.3.0
│ ├── 2.3.1
│ ├── 2.3.2
│ └── devTest of the pipeline
# Create output dir
mkdir nf-core-mag-test
# Load the modules
module load bioinfo-tools Nextflow/22.10.1 nf-core-pipelines/latest
# Set the variable for NXF_HOME
export NXF_HOME=/crex/proj/snic2020-6-222/Projects/Tconura/working/Andre/.nextflow
# Lets give it a test
nextflow run $NF_CORE_PIPELINES/mag/2.3.2/workflow -profile test,uppmax --project naiss2024-5-1 --outdir nf-core-mag-test/
N E X T F L O W ~ version 22.10.2
Launching `/sw/bioinfo/nf-core-pipelines/latest/rackham/mag/2.3.2/workflow/main.nf` [exotic_venter] DSL2 - revision: 8f9adbafb4
------------------------------------------------------
,--./,-.
___ __ __ __ ___ /,-._.--~'
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/mag v2.3.2
------------------------------------------------------
Core Nextflow options
runName : exotic_venter
containerEngine : singularity
launchDir : /crex/proj/snic2020-6-222/Projects/Tconura/working/Andre/CONURA_WGS
workDir : /crex/proj/snic2020-6-222/Projects/Tconura/working/Andre/CONURA_WGS/work
projectDir : /sw/bioinfo/nf-core-pipelines/latest/rackham/mag/2.3.2/workflow
userName : andbou
profile : test,uppmax
configFiles : /sw/bioinfo/nf-core-pipelines/latest/rackham/mag/2.3.2/workflow/nextflow.config
Input/output options
input : https://raw.githubusercontent.com/nf-core/test-datasets/mag/samplesheets/samplesheet.csv
outdir : nf-core-mag-test/
Reference genome options
igenomes_base : /sw/data/igenomes/
Institutional config options
custom_config_base : /sw/bioinfo/nf-core-pipelines/latest/rackham/mag/2.3.2/workflow/../configs/
config_profile_name : Test profile
config_profile_description : Minimal test dataset to check pipeline function
config_profile_contact : Phil Ewels (@ewels)
config_profile_url : https://www.uppmax.uu.se/
Max job request options
max_cpus : 2
max_memory : 6.GB
max_time : 6.h
Quality control for short reads options
phix_reference : /sw/bioinfo/nf-core-pipelines/latest/rackham/mag/2.3.2/workflow/assets/data/GCA_002596845.1_ASM259684v1_genomic.fna.gz
Quality control for long reads options
lambda_reference : /sw/bioinfo/nf-core-pipelines/latest/rackham/mag/2.3.2/workflow/assets/data/GCA_000840245.1_ViralProj14204_genomic.fna.gz
Taxonomic profiling options
centrifuge_db : https://raw.githubusercontent.com/nf-core/test-datasets/mag/test_data/minigut_cf.tar.gz
kraken2_db : https://raw.githubusercontent.com/nf-core/test-datasets/mag/test_data/minigut_kraken.tgz
skip_krona : true
gtdb : false
Binning options
skip_concoct : true
min_length_unbinned_contigs: 1
max_unbinned_contigs : 2
Bin quality check options
busco_reference : https://busco-data.ezlab.org/v5/data/lineages/bacteria_odb10.2020-03-06.tar.gz
busco_clean : true
checkm_db : null
gunc_db : null
!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use nf-core/mag for your analysis please cite:
* The pipeline publication
https://doi.org/10.1093/nargab/lqac007
* The pipeline
https://doi.org/10.5281/zenodo.3589527
* The nf-core framework
https://doi.org/10.1038/s41587-020-0439-x
* Software dependencies
https://github.com/nf-core/mag/blob/master/CITATIONS.md
------------------------------------------------------
WARN: Found unexpected parameters:
* --save_reference: true
- Ignore this warning: params.schema_ignore_params = "save_reference"
Unable to read script: '/sw/bioinfo/nf-core-pipelines/latest/rackham/mag/2.3.2/workflow/./workflows/mag.nf' -- cause: https://busco-data.ezlab.org/v5/data/lineages/bacteria_odb10.2020-03-06.tar.gz
Input specifications
To skip the qc and assembly step we can provide the assemblies and reads used to generate the assemblies in a CSV file together with a CSV file specifying the reads. The assembly CSV should be called with
--assembly_inputand the reads to be called with--input. Furthermore, these columns must be specified for the assembly csvid,group,assembler,fastawith one of these values for assembler: MEGAHIT, SPAdes, or SPAdesHybrid. The reads csv file must use these columns:sample,group,short_reads_1,short_reads_2,long_reads. Note that these files must contain the headersExample: samplesheet-assembly.csv
Example: samplesheet-reads.csv
Running the pipeline
Example command:
Updating the pipeline
Nextflow automatically pulls the pipeline code and stores it as a cached version. To make sure that you’re running the latest version of the pipeline, make sure that you regularly update the cached version of the pipeline:
nextflow pull nf-core/magReproducibily
Specify the pipeline version using
-r version. This will be logged in reports so one can always check what versions of the different tools were used.MetaBAT2is run by default with a fixed seed set in the pipeline.To keep
BUSCOreproducible, download a lineage beforehand as they are frequently updated and old versions are not always easily accessible.For the taxonomic bin classification with CAT, when running the pipeline with
--cat_db_generatethe parameter--save_cat_dbcan be used to also save the generated database to allow reproducibility in future runs. Note that when specifying a pre-built database with--cat_db, currently the database can not be saved.When it comes to visualizing taxonomic data using Krona, you have the option to provide a taxonomy file, such as taxonomy.tab, using the --krona_db parameter. If you don’t supply a taxonomy file, Krona is designed to automatically download the required taxonomy data for visualization.
The taxonomic classification of bins with GTDB-Tk is not guaranteed to be reproducible, since the placement of bins in the reference tree is non-deterministic. However, the authors of the GTDB-Tk article examined the reproducibility on a set of 100 genomes across 50 trials and did not observe any difference (see https://doi.org/10.1093/bioinformatics/btz848).
Note on bin refinement
DAS Toolmay not work as enough single-copy genes were not recovered. One can change the scoring threshold using-refine_bins_dastool_threshold, which will modify the scoring threshold defined in the DAS Tool publication.