saltstack-formulas / template-formula

SaltStack formula template filled with dummy content
http://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html
Other
119 stars 85 forks source link

[FEATURE] basic schema and data validation [cuelang] #232

Open noelmcloughlin opened 3 years ago

noelmcloughlin commented 3 years ago

Overview

There is no Data Constraints languages used in Saltstack formulas. If these were used, as part of CI/CD, then it would help formula developers, and provide example of how users could add/introduce data validation themselves. A related issue, is that no saltstack formula has a published schema, except what is available in README, pillar.example, etc.

To start a discussion I would suggest using Cuelang (https://cuelang.org/docs/install) to address three concerns:

DEMO

  1. install cuelang from https://cuelang.org/docs/install/

  2. goto a directory with YAML files

        $ git clone https://github.com/saltstack-formulas/template-formula
        $ cd template-formula/TEMPLATE/parameters/
  3. create a Schema file named schema.cue in this directory.

        #template: {
            pkg?: name?: string
            rootgroup?: string
            hide_output?: bool
            dir_mode?: =~"^0?[124567]{3}$" // any mode of length 3, with 0 prefix optional
            mode?: =~"^0?[124567]{3}$" // any mode of length 3, with 0 prefix optional
            config?:    string
            service?: name?: string
            subcomponent?: config: string
    
            // legacy
            pip_pkg?: string
            pkgs_add?: [...]
            pips?: [...]
    
            // Just here for testing
            added_in_defaults?: string
            winner?:            string
                ...
        }
        values?: {...#template}     // probable yaml namespace
  4. Make a change to some YAML file to introduce a bad value.

  5. Validate against schema data-constraints (cue vet FILE.yaml schema.cue)

            $ for f in $( find . | grep yaml$); do cue vet $f schema.cue; done
            values.pkg.name: conflicting values 111 and string (mismatched types int and string):
                .\os\Fedora.yaml:16:12
                .\schema.cue:10:18
                .\schema.cue:29:14
  6. Fix the error in your YAML and try again.

            $ for f in $( find . | grep yaml$); do cue vet $f schema.cue; done
            $ echo $?
             0

Related issues

Salt project: https://github.com/saltstack/salt/issues/54193

noelmcloughlin commented 3 years ago

@myii regarding Dhall. There is overlap with cue: https://github.com/cuelang/cue/discussions/669#discussioncomment-306811

My PR is to introduce basic Schema support without forcing cue. The help I need from you is to update your CI/CD image to install Cuelang (and Dhall perhaps). A basic pipeline job can check yaml against schema.cue schema (its only 1% coverage because I just wanted to showcase the possibility). The code footprint is tiny - one file.

A Dhall footprint could be introduced in different PR but neither dhall or cue should be forced.