Add rule engine for customized validation rules

[antonroman]

Currently there are three types of validation: syntax (just checks the descriptor file against its correspondent yaml schema), integrity (checks that all the used elements used are defined) and topology (checks that the forwarding graph is ). The aim of this issue is to add a fourth set of rules which can be defined before writing the actual descriptors. These rules do not verify that the descriptors are formally correct but they meet some specific requirements set beforehand by the NS developer or even the developers of the platform. If any change is added to the descriptors the validation tool will let know the developer if any of these rules is not met. Examples of these rules could be as follows:

1. Check that a VNF has a specific number of interfaces.
2. Check that a VNF has at least X interfaces of specific type.
3. Check connectivity types
4. Check that the VNF resources assigned to VNFs are above a minimum value.
5. Check that a NS has at least a X VNF of specific types.

For example, in the case of the VNF defined by Quobis we could set beforehand some rules like: "we always require a management interface for the WAC VNF".

The rules must be very easy to configure and implement, preferably in a know format, and they must be interpreted in rules engine within the validation tool. The type of supported rules should cover at least all the type of rules we have defined above but it should be as extensible as possible keeping it simple.

The use of these rules should be optional, only executed if requested when the validator is instantiated as for the rest of checks and the rule file is present.

These rules could also be used that the descriptors are compliant with the requirements of other platform (e.g. OSM), @mpeuster your feeback regarding this point would be appreciated.

We have identified these two rule engines which could be used:

• https://github.com/CacheControl/json-rules-engine
• https://github.com/venmo/business-rules Both of them define the rules in JSON, we have not found machine rules which uses YAML to define the rules- I understand that using YAML would be better as it's the format used for the descriptors.

[antonroman]

This is an initial of the solution we are going to implement to be able to add custom validation rules:

This is the rule engine we are planning to use: https://github.com/venmo/business-rules

The part which is still open is the set of variables that we are going to extract from the Descriptor YAML files in order to be able to apply the rules. We'll try to implement it in an extensible way keeping the rules as simple as possible.

The rules will be written in YAML and they will be translated to JSON (this should be doable in directly Python).

[antonroman]

This is an example of a JSON rule file to check the amount of CPU and RAM memory of a VNFD. For each rule it is necessary to define at least one condition and one action. To build the condition it will be possible to do comparison against values extracted from the VNFD in a variable list. Actions are also predefined and there will be initially two of them: raise_error and raise_warning, in order to notify the user validating the VNFD that there are conditions which are not compliant with the rules.

#Example of vairables which can be used in the rule file
MINIMAL_NUMBER_OF_CPUS = 2
FIRST_PATIENT_CONNECTION_FUP_SUBJECT = 'Email Subject'
FIRST_PATIENT_CONNECTION_FUP_TEMPLATE = 'first_pat_conntion.html'

RULES = [ 
    #first patient connection follow up 
    { 
        "conditions": { 
            "all": [ 
                { 
                    "name": "vdu_resource_requirements_cpu_vcpus", 
                    "operator": "equal_to_case_insensitive", 
                    "value": "MINIMAL_NUMBER_OF_CPUS", 
                }
            ]
        },
        "actions": [ 
            { 
                "name": "raise_error", 
                "params": { 
                    "error_message": 'Error in the VFD CPU number, please check it.'
                } 
            } 
        ]
    }
    { 
        "conditions": { 
            "all": [ 
                { 
                    "name": "vdu_resource_requirements_memory_size", 
                    "operator": "equal_to", 
                    "value": "2Gb" 
                } 
            ]
        },

        "actions": [ 
            { 
                "name": "raise_error", 
                "params": { 
                    "error_message": 'Error in the VFD RAM memory resources, please check it.' 
                } 
            } 
        ]
    }
]`

[mpeuster]

Hi @antonroman ... this design look very promising to me. I like the approach.

Regarding OSM: This will have to do with the point you already mentioned:

The part which is still open is the set of variables that we are going to extract from the Descriptor YAML files in order to be able to apply the rules. We'll try to implement it in an extensible way keeping the rules as simple as possible.

I think the rules themselves should be able to express on which fields of the descriptor YAML file they depend. With this, one could easily write rules for OSM without needing to modify the Python code of tng-sdk-validate.

If we look at an example (vcpu field in 5GTANGO vs OSM descriptors), both descirpors are similar but the field names and paths through the tree to the particular fields are different:

5GTANGO:

virtual_deployment_units:
  - id: "vdu01"
    vm_image: "http://registry.sonata-nfv.eu/html/files/VM_images/sonata-VM-2ports.qcow"
    vm_image_format: "qcow2"
    resource_requirements:
      cpu:
        vcpus: 1

Tree path is: virtual_deployment_units / resource_requirements /cpu / vcpus

OSM:

vnfd:vnfd-catalog:
    vnfd:
    -   id: hackfest1-vnf
        name: hackfest1-vnf
        short-name: hackfest1-vnf
        version: '1.0'
        description: A simple VNF descriptor w/ one VDU
        logo: osm.png
        connection-point:
        -   name: vnf-cp0
            type: VPORT
        vdu:
        -   id: hackfest1VM
            name: hackfest1VM
            image: "ubuntu:trusty"
            count: '1'
            vm-flavor:
                vcpu-count: '1'

Tree path is: vnfd:vnfd-catalog / vnfd / id: hackfest1-vnf / vdu / id: hackfest1VM / vm-flavor / vcpu-count

Maybe we could specify those tree paths to the fields we want to check in the rules themselves? Problem here are the specific names in the paths of the OSM descriptors like id: hackfest1VM. This makes everything a bit more complicated.

What would be your idea to handle this?

[antonroman]

This is an example of the rules which are already working in the beta implementation (pending to be merged):

# rule 1 - number of vCPUs
- conditions:
    all:
      - name: vdu_resource_requirements_cpu_vcpus
        operator: less_than
        value: 6
  actions:
    - name: raise_error
      params:
        error_text: The number of CPUs must be higher than 2!

# rule 2 - RAM size GB
- conditions:
    all:
      - name: vdu_resource_requirements_ram_size
        operator: less_than_or_equal_to
        value: 4
      - name: vdu_resource_requirements_ram_size_unit
        operator: equal_to
        value: "GB"
  actions:
    - name: raise_error
      params:
        error_text: The size of RAM must be higher than 4GB!

# rule 3 - RAM size MB
- conditions:
    all:
      - name: vdu_resource_requirements_ram_size
        operator: less_than_or_equal_to
        value: 4096
      - name: vdu_resource_requirements_ram_size_unit
        operator: equal_to
        value: "MB"
  actions:
    - name: raise_error
      params:
        error_text: The size of RAM must be higher than 4096MB!

# rule 4 - 
- conditions:
    all:
      - name: vdu_vm_resource_format
        operator: equal_to
        value: "docker"
  actions:
    - name: raise_warning
      params:
        error_text: Docker is not the recommended VM format, use a virtual machine instead

[antonroman]

@mpeuster thanks for your proposal. According to it the rules would look like this, right?:

For OSM:

# rule 1 - number of vCPUs
- conditions:
    all:
      - name: vdu_resource_requirements_cpu_vcpus
        tree-path: vnfd:vnfd-catalog / vnfd / id: hackfest1-vnf / vdu / id: hackfest1VM / vm-flavor / vcpu-count
        operator: less_than
        value: 2
  actions:
    - name: raise_error
      params:
        error_text: The number of CPUs must be higher than 2!

For 5GTANGO:

# rule 1 - number of vCPUs
- conditions:
    all:
      - name: vdu_resource_requirements_cpu_vcpus
        tree-path: virtual_deployment_units / resource_requirements /cpu / vcpus
        operator: less_than
        value: 2
  actions:
    - name: raise_error
      params:
        error_text: The number of CPUs must be higher than 2!

I think it's feasible, my only concern I see is that the developer must know in advance the name which is going to be given to the vdu to set the path tree, which does not happen in 5GTango.

In any case, the rules I will show in Athens won't include the path_tree param yet, so we can discuss this briefly there.