search

icgc-argo / workflow-management

GNU Affero General Public License v3.0
0 stars 1 forks source link

Update nextflow version to 22.10.7 #196

Open jaserud opened 2 years ago

jaserud commented 2 years ago

Detailed Description

Possible Implementation

Current Challenge

Nextflow new version has a bug, if we upgrade, our system will break. The older version affects running any WF.

Edmund is not blocked, he's on germline VC pipeline. Needs to run benchmarking test before needing the version upgrade.

Bug raised in nextflow: https://github.com/nextflow-io/nextflow/discussions/4019

Updates

Give them our PR, and see if they respond? Check if they have any contribution guide.

lindaxiang commented 1 year ago

We need to upgrade nextflow version to support running workflows which are written according to nf-core framework.
@Buwujiu @puneet-oicr Can you please prioritize the ticket. Thanks!

UmmulkiramR commented 1 year ago

works with the new version on QA now.

UmmulkiramR commented 1 year ago

The workflows triggered with the new version execute successfully the first time. For the subsequent runs, however, the library keeps throwing the IllegalArgumentException("Plugin system was already setup") exception.

After reaching out to the nextflow developers they requested for a github issue to be raised for this. Here's the github ticket I created - https://github.com/nextflow-io/nextflow/issues/4018

UmmulkiramR commented 1 year ago

The Nextflow team converted the 4018 bug report to a discussion. Here's the link -> https://github.com/nextflow-io/nextflow/discussions/4019 Requested an update again last week on this but no revert from them yet.

Buwujiu commented 1 year ago

can we make our own nextflow version and use it until they release a new version? We will fork the code, make the change, see if nextflow would approve.

UmmulkiramR commented 1 year ago

Management in dev and QA now uses the forked version of the nextflow library that contains the bug fix. PR created-> https://github.com/icgc-argo/workflow-management/pull/216

Works for both prealnqc and dnaalnqc workflows as tested by @lindaxiang here -> https://github.com/icgc-argo/workflow-roadmap/issues/388

UmmulkiramR commented 11 months ago

PR created on the original nextflow repo -> https://github.com/nextflow-io/nextflow/pull/4355 No revert on it yet.

UmmulkiramR commented 10 months ago

In the meantime, have forked the Nextflow library into our icgc-argo repo and added the proposed fix in there to unblock workflows in RDPC. We need to plan a release of WF management to prod that uses this forked version of the Nextflow library