CCBR / Dockers

A collection of Dockerfiles to facilitate reproducibility across computing environments.
11 stars 3 forks source link

consider docker release cycle #37

Open kelly-sovacool opened 3 weeks ago

kelly-sovacool commented 3 weeks ago

idea found by @kopardev: modifiers on tags for dev, feature, bug, main

how can we automate this to reduce maintenance overhead?

nextflow modules can use if/else statements to search for main/dev/feature tags

kopardev commented 2 weeks ago

@kelly-sovacool i was thinking of something like this

process exampleProcess {
    // Define the container images in priority order for both Singularity and Docker
    def containerPaths = [
        'xyz': [
            docker: 'xyz:v2'
        ],
        'xyzdev': [
            docker: 'xyz:v2-dev'
        ],
        'xyzfeat': [
           docker: 'xyz:v2-feat'
        ]
    ]

    // Find the first available container image
    def selectedImage = containerPaths.find { image, paths ->
        def testImage = workflow.containerEngine == 'singularity' ? paths.singularity : paths.docker
        "singularity pull ${testImage}".execute().waitFor() == 0
    }

    // Error handling if no image is found
    if (!selectedImage) {
        throw new RuntimeException("No suitable container image found.")
    }

    // Use the appropriate image based on the container engine
    container "${workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
        selectedImage.value.singularity :
        selectedImage.value.docker }"

    script:
    """
blah blah blah
    """
}

will this work? what do you think??

kelly-sovacool commented 2 weeks ago

@kopardev I think something like your code will work.

To reduce code repetition, I think we would want to put most of this code in a groovy function in lib/ (in the nextflow template repo), then the process definition could pass along the default container tag. So the structure would be something like:

modules/blah/example/main.nf:

process exampleProcess {

  container "${ Containers.select('xyz:v2') }"

  script:
  """
  ...
}

lib/Containers.groovy:

class Containers {
  public static String select(image) {
    // logic for selecting appropriate container goes here
    return image
  }
}