search

nf-core / differentialabundance

Differential abundance analysis for feature/ observation matrices from platforms such as RNA-seq
https://nf-co.re/differentialabundance
MIT License
57 stars 32 forks source link

DiffAbundance v1.4.0 does not work with most recent nextflow 23.10.1.5891 #237

Closed aghr closed 7 months ago

aghr commented 7 months ago

Description of the bug

After updating nextflow to the most recent version 23.10.1.5891 (Jan 12th, 2024) I cannot execute anymore diffabundance workflow v1.4.0 which I could execute before. The issue seems to be how parameters are handled. It looks that the recent nextflow does not accept the format how the diffabundance pipeline provides its parameters. Doing a quick test with the most recent NFcore rnaseq pipeline v3.14.0 it works with the most recent nextflow version, but I get the same error as I see with diffabundance v 1.4.0 when trying to run an older version like NFcore rnaseq 3.13.2 . It looks as if parts of the parameter handling need to be changed to be inline with the most recent nextflow version and that the NFcore rnaseq team did this when updating their pipeline to 3.14.0.

Command used and terminal output

nextflow run nf-core/differentialabundance  --input /XXX/samplesheet.csv     --study_name "Testrun"     --study_type "rnaseq"     --outdir $output_dir     --matrix /XXX/star_rsem/rsem.merged.gene_counts.tsv     --contrasts /XXX/contrasts.csv     --max_cpus 20     --max_memory 250GB     -profile singularity     -revision 1.4.0

#######################

Feb-20 17:21:50.966 [main] DEBUG nextflow.cli.CmdRun -
  Version: 23.10.1 build 5891
  Created: 12-01-2024 22:01 UTC (23:01 CEST)
  System: Linux 3.10.0-1062.18.1.el7.x86_64
  Runtime: Groovy 3.0.19 on OpenJDK 64-Bit Server VM 20.0.2-internal-adhoc..src
  Encoding: UTF-8 (ANSI_X3.4-1968)
  Process: 36817@scicomp-srv1.mpi-cbg.de [10.1.4.18]
  CPUs: 40 - Mem: 502.9 GB (466.8 GB) - Swap: 0 (0)
Feb-20 17:21:50.995 [main] DEBUG nextflow.Session - Work-dir: /XXX/work [lustre]
Feb-20 17:21:51.020 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Feb-20 17:21:51.032 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Feb-20 17:21:51.150 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Feb-20 17:21:51.162 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 41; maxThreads: 1000
Feb-20 17:21:51.275 [main] DEBUG nextflow.Session - Session start
Feb-20 17:21:51.280 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /projects/bioinfo/data/nfcore_degs/nfcore_differential_abundance/output_frozen_v1_4_0/pipeline_info/execution_trace_2024-02-20_17-21-50.txt
Feb-20 17:21:51.292 [main] DEBUG nextflow.Session - Using default localLib path: /home/kuecuek/.nextflow/assets/nf-core/differentialabundance/lib
Feb-20 17:21:51.297 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/kuecuek/.nextflow/assets/nf-core/differentialabundance/lib
Feb-20 17:21:51.298 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/kuecuek/.nextflow/assets/nf-core/differentialabundance/lib/nfcore_external_java_deps.jar
Feb-20 17:21:51.889 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Feb-20 17:21:51.910 [main] DEBUG nextflow.script.IncludeDef - Loading included plugin extensions with names: [validateParameters:validateParameters, paramsHelp:paramsHelp]; plugin Id: nf-validation
Feb-20 17:21:51.943 [main] DEBUG nextflow.validation.SchemaValidator - Starting parameters validation
Feb-20 17:21:52.046 [main] WARN  nextflow.validation.SchemaValidator - The following invalid input values have been detected:

* --input: /XXX/samplesheet.csv
* --study_name: Testrun
* --study_type: rnaseq
* --study_abundance_type: counts
* --contrasts: /XXX/contrasts.csv
* --querygse: null
* --matrix: /XXX/star_rsem/rsem.merged.gene_counts.tsv
* --transcript_length_matrix: null
* --control_features: null
* --sizefactors_from_controls: false
* --logo_file: /XXX/nf-core-differentialabundance_logo_light.png
* --css_file: /XXX/nf-core_style.css
* --citations_file: /XXX/CITATIONS.md
* --report_file: /XXX/differentialabundance_report.Rmd
* --report_title: null
* --report_author: null
* --report_description: null
* --report_scree: true
* --observations_type: sample
* --observations_id_col: sample
* --observations_name_col: sample
* --features: null
* --features_type: gene
* --features_id_col: gene_id
* --features_name_col: gene_name
* --features_metadata_cols: gene_id,gene_name,gene_biotype
* --features_log2_assays: null
* --features_gtf_feature_type: transcript
* --features_gtf_table_first_field: gene_id
* --affy_cel_files_archive: null
* --affy_file_name_col: file
* --features_metadata_cols: gene_id,gene_name,gene_biotype
* --features_log2_assays: null
* --features_gtf_feature_type: transcript
* --features_gtf_table_first_field: gene_id
* --affy_cel_files_archive: null
* --affy_file_name_col: file
* --affy_background: true
* --affy_bgversion: 2
* --affy_destructive: false
* --affy_cdfname: null
* --affy_rm_mask: false
* --affy_rm_outliers: false
* --affy_rm_extra: false
* --affy_build_annotation: true
* --proteus_measurecol_prefix: LFQ intensity
* --proteus_norm_function: normalizeMedian
* --proteus_plotsd_method: violin
* --proteus_plotmv_loess: true
* --proteus_palette_name: Set1
* --proteus_round_digits: -1
* --filtering_min_samples: 1
* --filtering_min_abundance: 1
* --filtering_min_proportion: null
* --filtering_grouping_var: null
* --exploratory_main_variable: auto_pca
* --exploratory_clustering_method: ward.D2
* --exploratory_cor_method: spearman
* --exploratory_n_features: 500
* --exploratory_whisker_distance: 1.5
* --exploratory_mad_threshold: -5
* --exploratory_assay_names: raw,normalised,variance_stabilised
* --exploratory_final_assay: variance_stabilised
* --exploratory_palette_name: Set1
* --differential_file_suffix: .deseq2.results.tsv
* --differential_feature_id_column: gene_id
* --differential_feature_name_column: gene_name
* --differential_fc_column: log2FoldChange
* --differential_pval_column: pvalue
* --differential_qval_column: padj
* --differential_min_fold_change: 2
* --differential_max_pval: 1
* --differential_max_qval: 0.05
* --differential_foldchanges_logged: true
* --differential_palette_name: Set1
* --differential_subset_to_contrast_samples: false
* --deseq2_test: Wald
* --deseq2_fit_type: parametric
* --deseq2_sf_type: ratio
* --deseq2_min_replicates_for_replace: 7
* --deseq2_use_t: false
* --deseq2_lfc_threshold: 0
* --deseq2_alt_hypothesis: greaterAbs
* --deseq2_independent_filtering: true
* --deseq2_p_adjust_method: BH
* --deseq2_alpha: 0.1
* --deseq2_minmu: 0.5
* --deseq2_vs_method: vst
* --deseq2_shrink_lfc: true
* --deseq2_cores: 1
* --deseq2_vs_blind: true
* --deseq2_vst_nsub: 1000
* --limma_ndups: null
* --limma_spacing: null
* --limma_block: null
* --limma_correlation: null
* --limma_method: ls
* --limma_proportion: 0.01
* --limma_stdev_coef_lim: 0.1,4
* --limma_trend: false
* --limma_robust: false
* --limma_winsor_tail_p: 0.05,0.1
* --limma_adjust_method: BH
* --limma_p_value: 1
* --limma_lfc: 0
* --limma_confint: false
* --gsea_run: false
* --gsea_nperm: 1000
* --gsea_permute: phenotype
* --gsea_scoring_scheme: weighted
* --gsea_metric: Signal2Noise
* --gsea_sort: real
* --gsea_order: descending
* --gsea_set_max: 500
* --gsea_set_min: 15
* --gsea_norm: meandiv
* --gsea_rnd_type: no_balance
* --gsea_make_sets: true
* --gsea_median: false
* --gsea_num: 100
* --gsea_plot_top_x: 20
* --gsea_rnd_seed: timestamp
* --gsea_save_rnd_lists: false
* --gsea_zip_report: false
* --gsea_gene_sets: null
* --shinyngs_build_app: true
* --shinyngs_guess_unlog_matrices: true
* --shinyngs_deploy_to_shinyapps_io: false
* --shinyngs_shinyapps_account: null
* --shinyngs_shinyapps_app_name: null
* --genome: null
* --igenomes_ignore: false
* --outdir: /XXX/
* --publish_dir_mode: copy
* --email: null
* --email_on_fail: null
* --plaintext_email: false
* --hook_url: null
* --version: false
* --config_profile_name: null
* --config_profile_description: null
* --custom_config_version: master
* --max_cpus: 20
* --max_memory: 250GB
* --custom_config_base: https://raw.githubusercontent.com/nf-core/configs/master
* --config_profile_contact: null
* --config_profile_url: null
* --max_time: 240.h
* --validate_params: true
* --gtf: null

Feb-20 17:21:52.061 [main] ERROR n.validation.JsonSchemaValidator - Failed to load the meta schema:
The used schema draft (http://json-schema.org/draft-07/schema) is not correct, please use "https://json-schema.org/draft/2020-12/schema" instead.
See here for more information: https://json-schema.org/specification#migrating-from-older-drafts

Feb-20 17:21:52.070 [main] DEBUG nextflow.script.ScriptRunner - Parsed script files:
  Script_66ffa6df5b55d922: /home/kuecuek/.nextflow/assets/nf-core/differentialabundance/main.nf
Feb-20 17:21:52.070 [main] DEBUG nextflow.Session - Session aborted -- Cause: nextflow.validation.SchemaValidationException:
Feb-20 17:21:52.078 [main] DEBUG nextflow.cli.Launcher - Operation aborted
nextflow.validation.SchemaValidationException:
        at nextflow.validation.JsonSchemaValidator.validateObject(JsonSchemaValidator.groovy:43)
        at nextflow.validation.JsonSchemaValidator.validate(JsonSchemaValidator.groovy:109)
        at nextflow.validation.SchemaValidator.validateParameters(SchemaValidator.groovy:344)
        at nextflow.validation.SchemaValidator.validateParameters(SchemaValidator.groovy)
       at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
        at java.base/java.lang.reflect.Method.invoke(Method.java:578)
        at nextflow.script.FunctionDef.invoke_a(FunctionDef.groovy:64)
        at nextflow.script.ComponentDef.invoke_o(ComponentDef.groovy:40)
        at nextflow.script.WorkflowBinding.invokeMethod(WorkflowBinding.groovy:102)
        at groovy.lang.GroovyObject$invokeMethod.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:148)
        at nextflow.script.BaseScript.invokeMethod(BaseScript.groovy:140)
        at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:68)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:51)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:171)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:176)
        at Script_66ffa6df5b55d922.runScript(Script_66ffa6df5b55d922:41)
        at nextflow.script.BaseScript.run0(BaseScript.groovy:144)
        at nextflow.script.BaseScript.run(BaseScript.groovy:192)
        at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:236)
        at nextflow.script.ScriptRunner.run(ScriptRunner.groovy:242)
        at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:137)
        at nextflow.cli.CmdRun.run(CmdRun.groovy:372)
        at nextflow.cli.Launcher.run(Launcher.groovy:500)
        at nextflow.cli.Launcher.main(Launcher.groovy:672)

Relevant files

No response

System information

Nextflow version 23.10.1 build 5891 created 12-01-2024 22:01 UTC (23:01 CEST) Desktop Executor: local Singularity OS: CentOS Linux release 7.7.1908 Version of nf-core/differentialabundance: 1.4.0

aghr commented 7 months ago

Running the pipeline with the very same parameters with nextflow 23.10.0 works.

WackerO commented 7 months ago

Ah yes, I'll have to pin the validation version, this also happened to me in another pipeline.

Edit: Actually I just saw that the plugin is already pinned on the dev branch; can you try rerunning with that?

WackerO commented 7 months ago

As the dev branch already pins the validator version, I'll close this issue for now. If the error also appears while running the dev pipeline, please feel free to reopen