search

galaxyproject / galaxy-helm

Minimal setup required to run Galaxy under Kubernetes
MIT License
42 stars 39 forks source link

Make it easier to override tool mappings #171

Closed nuwang closed 3 years ago

nuwang commented 4 years ago

container_mapper_rules.yml allows you to manually remap tools to specific container overrides, or reassign their default resource allocations. However, because the rules are structured as an array, it's difficult to override the defaults in the helm chart when deploying in production. Therefore, turning it into a key based structure would greatly simplify administration in practice:

For example:

     mappings:
         - tool_ids:
            - toolshed.g2.bx.psu.edu/repos/iuc/jbrowse/jbrowse/1.16.5+galaxy6
          container:
            docker_container_id_override: cloudve/jbrowse:1.16.5
        - tool_ids:
            - sort1
            - Grouping1
          container:
            docker_container_id_override: {{ .Values.image.repository }}:{{ .Values.image.tag }}
            resource_set: small
        - tool_ids:
            - toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/.*
            - toolshed.g2.bx.psu.edu/repos/iuc/bwameth/bwameth/.*
            - toolshed.g2.bx.psu.edu/repos/iuc/featurecounts/featurecounts/.*
            - toolshed.g2.bx.psu.edu/repos/iuc/hisat2/hisat2/.*
            - toolshed.g2.bx.psu.edu/repos/iuc/valet/valet/.*
            - toolshed.g2.bx.psu.edu/repos/iuc/varscan_somatic/varscan_somatic/.*
            - toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_bam2wig/.*
          container:
            resource_set: medium

Would become something like:

     mappings:
        jbrowse:
          tool_ids:
            - toolshed.g2.bx.psu.edu/repos/iuc/jbrowse/jbrowse/1.16.5+galaxy6
          container:
            docker_container_id_override: cloudve/jbrowse:1.16.5
        galaxy_tools:
          tool_ids:
            - sort1
            - Grouping1
          container:
            docker_container_id_override: {{ .Values.image.repository }}:{{ .Values.image.tag }}
            resource_set: small
        medium_group:
           tool_ids:
            - toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/.*
            - toolshed.g2.bx.psu.edu/repos/iuc/bwameth/bwameth/.*
            - toolshed.g2.bx.psu.edu/repos/iuc/featurecounts/featurecounts/.*
            - toolshed.g2.bx.psu.edu/repos/iuc/hisat2/hisat2/.*
            - toolshed.g2.bx.psu.edu/repos/iuc/valet/valet/.*
            - toolshed.g2.bx.psu.edu/repos/iuc/varscan_somatic/varscan_somatic/.*
            - toolshed.g2.bx.psu.edu/repos/nilesh/rseqc/rseqc_bam2wig/.*
          container:
            resource_set: medium
almahmoud commented 4 years ago

I think so far, we've settled on

    galaxy_group:
        tool_ids:
            original_tools:
                - Summary_Statistics1
                - xyz
            additional_tools:
                - new_tool
        docker_container_id_override: cloudve/gsummary:latest
        resource_set: small
        extra_specs:
            k8s_node_selector: whatever
            k8s_service_account: whatever

as a quick change mainly focusing on easily allowing 1) edits via helm and 2) injecting new tools to already existing groups via a helm.

Friends with https://github.com/galaxyproject/galaxy/issues/10481