birchb1024 / yamp

Yet Another Macro Processor - for YAML - Superseded by Goyamp
GNU General Public License v3.0
13 stars 3 forks source link

Feature Request: ifdef directive starts a section of conditionally compiled code #11

Closed kingdase closed 5 years ago

kingdase commented 5 years ago

It would be great to use ifdef to remove whole blocks in yaml. I tried the if statement, but it does not work.

We have code blocks in our concourse pipeline that involve staging, and they're not needed in production stage because it's the final stage.

We use the same pipeline code for all stages.

As a workaround I use begin / end comments and suppress the enclosing lines with awk before calling yamp.

the ifdef statement should, if possible, query the environemnt.

e.g.

ifdef: { python_eval: 'env["stage"]  != "prod"' }
...
endif

Enclosed a section of our pipeline with the mentioned comments

---
groups:
- name: pcf-automation
  jobs:
    flatten:
      - get-new-resources
      - check-runned-om-installation-pipelinetask-opsmgr
      - check-runned-backup-ert-pipelinetask-opsmgr
      - check-runned-backup-director-pipelinetask-opsmgr
      - create-infrastructure
      - run-p-smoketests
      - pause-pipelines
      - unpause-pipelines
      - configure-director
      - apply-product-changes
      - run-p-smoketest-after-deployment
      - upload-stemcells

# START NOT-IN-PROD
      - promote-opsmgr
      - repeat:
          for: tile
          in: stems
          body:
            - promote-{{tile.name}}

      - promote-platform-automation
      - promote-pas
# END NOT-IN-PROD

      - upgrade-opsman
      - extend-opsman-vm-types

      - upload-and-stage-pas
      - configure-pas

      - repeat:
          for: tile
          in: tilex
          body:
            - upload-and-stage-{{tile.name}}
            - configure-{{tile.name}}
# START NOT-IN-PROD
            - promote-{{tile.name}}
# END NOT-IN-PROD

- name: config-and-maintenance
  jobs:
  - start-all-pcf-automation-tasks
# START NOT-IN-PROD
  - promote-all
# END NOT-IN-PROD
  - create-infrastructure-manual

resource_types:
- name: pivnet
  type: docker-image
  source:
    repository: pivotalcf/pivnet-resource
    tag: latest-final
birchb1024 commented 5 years ago

Hi,

Thanks for the report and the idea. Actually I found the 'if' statement seems to work. I attach below the working example. You could try it this way and remove the awk?

Perhaps the feature could be 'skip current document'. ie processing would continue at the next document (after ---). ie

---
document_skip_if: $PROD
#.. rest is skipped
---
# processing resumes here

That way whole blocks could be removed. But this wont solve your actual use case. (But 'if' alone seems to..)

Here's that code

#
# Usage on Linux: 
#
#    $ ENVIRON=test python src/yamp.py test/regression/issue11.yaml
#
# Example executions:
#
# $ ENVIRON=test python src/yamp.py test/regression/issue11.yaml > /tmp/test.yaml
# $ ENVIRON=production python src/yamp.py test/regression/issue11.yaml > /tmp/prod.yaml
# $ diff /tmp/prod.yaml /tmp/test.yaml
# 2c2
# < Generating for env 'production'. The $NON_PROD flag is False
# ---
# > Generating for env 'test'. The $NON_PROD flag is True
# 18a19,22
# >   - promote-opsmgr
# >   - promote-STEM-1
# >   - promote-platform-automation
# >   - promote-pas
# 24a29
# >   - promote-TILEX-1
# 27a33
# >   - promote-all

---
# First simulate some of the data this script seems to require...
- define: 
    stems:  [ { name: STEM-1 } ]
    tilex: [ {name: TILEX-1 } ]
    $ENVIRONMENT: env.ENVIRON
---
# I'm adding an environment configuration data chunk now.
# The 'production' nature is an attribute of the environment, 
# modelling this in data...
define:
  environments:
    test: 
      isProduction: false
      description: The default non-production test environment
    production: 
      isProduction: true
      description: The Production Environment
---
# Now we do some house keeping the set a global flag if this is a non-prod
# this keeps the template a bit simpler..        
- if: environments.$ENVIRONMENT.isProduction 
  then: {define: {$NON_PROD: false}}
  else: {define: {$NON_PROD: true}}
---
# Brag about what we know and how the flag is set
"Generating for env '{{$ENVIRONMENT}}'. The $NON_PROD flag is {{$NON_PROD}}" 
---
groups:
- name: pcf-automation
  jobs:
    flatten:
      - get-new-resources
      - check-runned-om-installation-pipelinetask-opsmgr
      - check-runned-backup-ert-pipelinetask-opsmgr
      - check-runned-backup-director-pipelinetask-opsmgr
      - create-infrastructure
      - run-p-smoketests
      - pause-pipelines
      - unpause-pipelines
      - configure-director
      - apply-product-changes
      - run-p-smoketest-after-deployment
      - upload-stemcells
# START NOT-IN-PROD
      - if: $NON_PROD
        then:
          - promote-opsmgr
          - repeat:
              for: tile
              in: stems
              body:
                - promote-{{tile.name}}

          - promote-platform-automation
          - promote-pas
# END NOT-IN-PROD

      - upgrade-opsman
      - extend-opsman-vm-types

      - upload-and-stage-pas
      - configure-pas

      - repeat:
          for: tile
          in: tilex
          body:
            - upload-and-stage-{{tile.name}}
            - configure-{{tile.name}}
# START NOT-IN-PROD
            - if: $NON_PROD
              then: promote-{{tile.name}}
# END NOT-IN-PROD

- name: config-and-maintenance
  jobs:
  - start-all-pcf-automation-tasks
# START NOT-IN-PROD
  - if: $NON_PROD
    then: promote-all
# END NOT-IN-PROD
  - create-infrastructure-manual

resource_types:
- name: pivnet
  type: docker-image
  source:
    repository: pivotalcf/pivnet-resource
    tag: latest-final
kingdase commented 5 years ago

Hi Peter,

thany you very much for your help !

I’m just playing around with your example code.

Thx

Jens