nextflow-io / nf-nomad

Hashicorp Nomad executor plugin for Nextflow

https://nextflow-io.github.io/nf-nomad/

Apache License 2.0

2 stars 3 forks source link

Roadmap v0.1.0 #2

Closed abhi18av closed 8 months ago

abhi18av commented 1 year ago

Assumptions for the plugin

docker executor is available on all nodes
A shared file system is present across the nomad nodes, with a persistent volume
A prod setup of a minimal of two nodes -> server node and client node

BONUS: Internet connection and wave enabled for fusionfs (?)

Pipelines to be tested on nomad cluster setup

nextflow-io/hello
rnaseq-nf
nf-core/fetchngs
nf-core/rnaseq
MAGMA

Plugin dev group (Jorge, Abhinav) responsible for

Developing the plugin structure and local testing setups via a nomad -dev node - #4
Making sure the Nextflow abstractions translate well to the nomad job abstractions #5
Testing the standard pipelines for usability and a smooth Nextflow experience locally

Infrastructure group (Tom, Matthias) responsible for

Setting up a common nomad testing infra (liasoning with nf-core team)
Setting up the testing infrastructure for pipelines via Github Actions (+ local runner) to be triggered on dev/main branch release/merges #3
Coming up with the Nextflow config for nomad executor

Long term maintenance

Jorge and Abhinav invested as developers
CMGG invested in maintaining the nf-core / Nextflow and Nomad infrastructure

abhi18av commented 1 year ago

@matthdsm @tomiles , the issues are now active and allocated for iteration-3.

abhi18av commented 8 months ago

I'm marking this issue as closed since the individual tasks would be added to the milestones.