Add e2e support for setting backend via Policy

[mszostok]

Description

Changes proposed in this pull request:

Changes

• Logger aware Argo Renderer

• Update CRD:

  output:
  typeInstances:
    backend:
    id: "123" # backend ref in which TI is stored
    abstract: true/false # whether ref. backend is abstract or not

  • Propagate backend data into TypeInstance output under Action CRD

• Update Engine GraphQL to show backend information for output TypeInstance

  • update Go clients
  • add unit-test coverage

• Update Local Hub GraphQL schemas and examples.

  • Add TypeInstanceBackend and handle it properly in Cypher. This is a draft implementation but already gives us a huge benefit as it enabled an option to create e2e test and protect current contract. If we rewrite/refactor Local Hub, we can protect ourselves from regressions.

• Ensure that core storage is registered when Local Hub starts.

• Add support for setting backend based on TypeRefs via Global Policy and Action Policy (support custom merging):

  typeInstance:
  rules:
    - typeRef:
        path: "cap.type.aws.auth.credentials"
        revision: "0.1.0" # optional, if not provided, applies to any revision
      backend:
        id: "00fd161c-01bd-47a6-9872-47490e11f996" # Vault backend storage

  • Pattern matching:

    // GetByTypeRef returns storage backend for a given TypeRef.
    // If backend for an explicit TypeRef is not found, the pattern matching is used.
    //
    // For example, if TypeRef is `cap.type.capactio.examples.message:0.1.0`:
    // - cap.type.capactio.examples.*:0.1.0
    // - cap.type.capactio.examples.*
    // - cap.type.capactio.*:0.1.0
    // - cap.type.capactio.*
    // - cap.type.*:0.1.0
    // - cap.type.*
    // - cap.*:0.1.0
    // - cap.*
    //

  • add Policy validation.

• Extends workflow syntax with capact-outputTypeInstance[*].backend

• Update Argo renderer to support backends based on TypeRef or capact-outputTypeInstance[*].backend alias. Once rendered the delegated backend storage is deterministic as it's "hardcoded" to a given ID.

• Add integration test for: Success:

  1. Create action with default storage
     • Empty Policy -> uses builtin storage
     • Policy with TypeRef defaults -> uses a given backend ID
  2. Create action with Helm storage (requires)
     • Policy with requires under interface.
     • Policy with default requires + override with interface specific requires (Will be done in follow-up PR when global default will be implemented)

Other:

• now make dev-cluster(-update) accepts COMPONENTS and BUILD_IMAGES_LIST envs. TODO: update dev doc on website.
• fix setting Hub manifests during capact install.

Testing

Integration tests prove that it works but here is also a “manual” opt to run a few examples:

1. Build Capact CLI: make build-tool-cli and use for all below commands.

2. Create TypeInstances for update and download:

   cat > /tmp/download-ti.yaml << ENDOFFILE
   typeInstances:
     - alias: "download"
       typeRef:
         path: cap.type.capactio.capact.validation.download
         revision: 0.1.0
       value:
         key: true
   ENDOFFILE
   export DOWNLOAD_TI=$(capact ti create -f /tmp/download-ti.yaml -ojson | jq -r '.[] | select(.alias == "download") | .id')
   cat > /tmp/update-ti.yaml << ENDOFFILE
   typeInstances:
     - alias: "update"
       typeRef:
         path: cap.type.capactio.capact.validation.update
         revision: 0.1.0
       value:
         key: true
   ENDOFFILE
   export UPDATE_TI=$(capact ti create -f /tmp/update-ti.yaml -ojson | jq -r '.[] | select(.alias == "update") | .id')

3. Create Helm storage TypeInstance:

   cat > /tmp/helm-storage.yaml << ENDOFFILE
   typeInstances:
     - alias: "helm"
       typeRef:
         path: cap.type.helm.storage
         revision: 0.1.0
       value:
         url: "ever" # currently, not needed
         acceptValues: false # just for demo purposes
   ENDOFFILE
   export HELM_STORAGE_TI=$(capact ti create -f /tmp/helm-storage.yaml -ojson | jq -r '.[] | select(.alias == "helm") | .id')

Scenarios

Success:

1. Create action with default (built-in) storage as Policy is empty and Implementation doesn't use any required backend.

   1. Create Action input:

      cat > /tmp/act-input-ti.yaml << ENDOFFILE
      typeInstances:
        - name: "testUpdate"
          id: ${UPDATE_TI}
        - name: "testInput"
          id: ${DOWNLOAD_TI}
      ENDOFFILE

   2. Run Action

      capact act create cap.interface.capactio.capact.validation.action.passing --name test --type-instances-from-file /tmp/act-input-ti.yaml

          `capact act run & capact act watch`

   3. Get Action output TypeInstances:

      capact act get test -ojson | jq '.Actions[0].output.typeInstances'

      Example output:

      [
        {
          "backend": {
            "abstract": true,
            "id": "318b99bd-9b26-4bc1-8259-0a7ff5dae61c"
          },
          "id": "12d18143-49b9-4604-9a4e-eb407ca76c78",
          "typeRef": {
            "path": "cap.type.capactio.capact.validation.upload",
            "revision": "0.1.0"
          }
        },
        {
          "backend": {
            "abstract": true,
            "id": "318b99bd-9b26-4bc1-8259-0a7ff5dae61c"
          },
          "id": "b06c97bc-0546-450f-a34d-8f38966dbea0",
          "typeRef": {
            "path": "cap.type.capactio.capact.validation.update",
            "revision": "0.1.0"
          }
        }
      ]

2. Create action with default storage based on typeinstace Policy.

   1. Update Global Policy:

      cat > /tmp/type-ref-policy.yaml << ENDOFFILE
      interface:
        rules:
        - interface:
            path: cap.*
          oneOf:
          - implementationConstraints:
              requires:
              - path: cap.core.type.platform.kubernetes
          - implementationConstraints: {}
      typeInstance:
        rules:
          - typeRef:
              path: cap.type.capactio.capact.validation.upload
              revision: 0.1.0
            backend:
              id: ${HELM_STORAGE_TI}
              description: Default Hub backend storage via TypeRef
      ENDOFFILE
      capact policy apply -f /tmp/type-ref-policy.yaml

   2. Create Action input:

      cat > /tmp/act-input-ti.yaml << ENDOFFILE
      typeInstances:
        - name: "testUpdate"
          id: ${UPDATE_TI}
        - name: "testInput"
          id: ${DOWNLOAD_TI}
      ENDOFFILE

   3. Run Action

      capact act create cap.interface.capactio.capact.validation.action.passing --name test-type-ref --type-instances-from-file /tmp/act-input-ti.yaml

          `capact act run & capact act watch`

   4. Get Action output TypeInstances:

      capact act get test-type-ref -ojson | jq '.Actions[0].output.typeInstances'

      Example output:

      [
        {
          "backend": {
            "abstract": false,
            "id": "e9d424b3-c55d-4b02-9e92-9c736b96f98e"
          },
          "id": "4fd10ff0-5eca-46db-b95e-3c8bab3767ba",
          "typeRef": {
            "path": "cap.type.capactio.capact.validation.upload",
            "revision": "0.1.0"
          }
        },
        {
          "backend": {
            "abstract": true,
            "id": "318b99bd-9b26-4bc1-8259-0a7ff5dae61c"
          },
          "id": "b06c97bc-0546-450f-a34d-8f38966dbea0",
          "typeRef": {
            "path": "cap.type.capactio.capact.validation.update",
            "revision": "0.1.0"
          }
        }
      ]

3. Create Action with Helm storage, which is enforced in Implementation requires section.

   1. Create required TypeInstance:

      cat > /tmp/inject-ti.yaml << ENDOFFILE
      typeInstances:
        - alias: "inject"
          typeRef:
            path: cap.type.capactio.capact.validation.single-key
            revision: 0.1.0
          value:
            key: true
      ENDOFFILE
      export INJECT_TI=$(capact ti create -f /tmp/inject-ti.yaml -ojson | jq -r '.[] | select(.alias == "inject") | .id')

   2. Update Global Policy:

      cat > /tmp/inject-storage-policy.yaml << ENDOFFILE
      interface:
        rules:
        - interface:
            path: cap.interface.capactio.capact.validation.action.passing
          oneOf:
          - implementationConstraints:
              attributes:
              - path: cap.attribute.capactio.capact.validation.policy.most-preferred
              requires:
              - path: cap.type.capactio.capact.validation.single-key
            inject:
              requiredTypeInstances:
              - description: Test TypeInstance
                id: ${INJECT_TI}
              - description: Helm backend TypeInstance
                id: ${HELM_STORAGE_TI}
          - implementationConstraints:
              path: cap.implementation.capactio.capact.validation.action.passing-a
        - interface:
            path: cap.*
          oneOf:
          - implementationConstraints: {}
      typeInstance:
        rules: []
      ENDOFFILE
      capact policy apply -f /tmp/inject-storage-policy.yaml

   3. Create Action input:

      cat > /tmp/act-input-ti.yaml << ENDOFFILE
      typeInstances:
        - name: "testUpdate"
          id: ${UPDATE_TI}
        - name: "testInput"
          id: ${DOWNLOAD_TI}
      ENDOFFILE

   4. Run Action

      capact act create cap.interface.capactio.capact.validation.action.passing --name inject-storage --type-instances-from-file /tmp/act-input-ti.yaml

      capact act run & capact act watch

   5. Get Action output TypeInstances:

      capact act get inject-storage -ojson | jq '.Actions[0].output.typeInstances'

      Example output:

      [
        {
          "backend": {
            "abstract": false,
            "id": "e9d424b3-c55d-4b02-9e92-9c736b96f98e"
          },
          "id": "cdf4ff38-f66e-4d03-a6ed-1093f154d3af",
          "typeRef": {
            "path": "cap.type.capactio.capact.validation.upload",
            "revision": "0.1.0"
          }
        }
      ]

Check with capact ti get that uses were properly set for a given Storage Backends.

Above scenarios can be repeated with Action Policy which will override the Global Policy.

Failures:

1. Create action with wrong type as storage for TypeRef

   cat > /tmp/type-ref-policy.yaml << ENDOFFILE
   interface:
     rules:
     - interface:
         path: cap.*
       oneOf:
       - implementationConstraints:
           requires:
           - path: cap.core.type.platform.kubernetes
       - implementationConstraints: {}
   typeInstance:
     rules:
       - typeRef:
           path: cap.type.capactio.capact.validation.upload
           revision: 0.1.0
         backend:
           id: ${DOWNLOAD_TI}
           description: Default Hub backend storage via TypeRef
   ENDOFFILE
   capact policy apply -f /tmp/type-ref-policy.yaml

   capact act create cap.interface.capactio.capact.validation.action.passing --name test-v1 --type-instances-from-file /tmp/act-input-ti.yaml

   capact act get test-v1 -oyaml

   Expected output:

       message: |-
         Cannot render given action: while rendering Action: while listing ImplementationRevisions for Interface "cap.interface.capactio.capact.validation.action.passing:": while TypeInstance metadata validation after resolving TypeRefs:
         - Metadata for "BackendTypeInstance":
             * Type reference ID: "2f856900-1b03-488e-8212-f7d86c2ae3e2", description: "Default Hub backend storage via TypeRef" is not a Hub storage (will retry - 10/15)

2. Create action without storage injected as requires part

   1. Update Policy:

      cat > /tmp/inject-storage-policy.yaml << ENDOFFILE
      interface:
      rules:
      - interface:
        path: cap.interface.capactio.capact.validation.action.passing
      oneOf:
      - implementationConstraints:
          attributes:
          - path: cap.attribute.capactio.capact.validation.policy.most-preferred
          requires:
          - path: cap.type.capactio.capact.validation.single-key
        inject:
          requiredTypeInstances:
          - description: Test TypeInstance
            id: ${INJECT_TI}
      typeInstance:
      rules: []
      ENDOFFILE
      capact policy apply -f /tmp/inject-storage-policy.yaml

      1. Create action:

         capact act create cap.interface.capactio.capact.validation.action.passing --name test-v2 --type-instances-from-file /tmp/act-input-ti.yaml

   2. Get output:

      capact act get test-v2 -oyaml

      Expected output:

      message: 'Cannot render given action: while rendering Action: while picking ImplementationRevision
        for Interface "cap.interface.capactio.capact.validation.action.passing:": No
        Implementations found with current policy for given Interface (will retry -
        3/15)'

3. Create action with wrong type as storage injected as requires part

   1. Update Policy

      cat > /tmp/inject-storage-policy.yaml << ENDOFFILE
      interface:
      rules:
      - interface:
        path: cap.interface.capactio.capact.validation.action.passing
      oneOf:
      - implementationConstraints:
          attributes:
          - path: cap.attribute.capactio.capact.validation.policy.most-preferred
          requires:
          - path: cap.type.capactio.capact.validation.single-key
        inject:
          requiredTypeInstances:
          - description: Test TypeInstance
            id: ${INJECT_TI}
          - description: Helm backend TypeInstance
            id: ${DOWNLOAD_TI}
      ENDOFFILE
      capact policy apply -f /tmp/inject-storage-policy.yaml

      1. Create action:

         capact act create cap.interface.capactio.capact.validation.action.passing --name test-v3 --type-instances-from-file /tmp/act-input-ti.yaml

   2. Get output:

      capact act get test-v3 -oyaml

      Expected output:

      message: 'Cannot render given action: while rendering Action: while picking ImplementationRevision
        for Interface "cap.interface.capactio.capact.validation.action.passing:": No
        Implementations found with current policy for given Interface (will retry -
        3/15)'

4. Wrong capact-outputTypeInstances[].backend type.

   1. First change the backend to injected in https://github.com/mszostok/os-hub-manifests/blob/policy-syntax/type-instances/manifests/implementation/capactio/capact/validation/action/passing-b.yaml#L73. Next, populate updated manifests.
   2. Update Policy:

      cat > /tmp/inject-storage-policy.yaml << ENDOFFILE
      interface:
        rules:
        - interface:
            path: cap.interface.capactio.capact.validation.action.passing
          oneOf:
          - implementationConstraints:
              attributes:
              - path: cap.attribute.capactio.capact.validation.policy.most-preferred
              requires:
              - path: cap.type.capactio.capact.validation.single-key
            inject:
              requiredTypeInstances:
              - description: Test TypeInstance
                id: ${INJECT_TI}
              - description: Helm backend TypeInstance
                id: ${HELM_STORAGE_TI}
          - implementationConstraints:
              path: cap.implementation.capactio.capact.validation.action.passing-a
        - interface:
            path: cap.*
          oneOf:
          - implementationConstraints: {}
      typeInstance:
        rules: []
      ENDOFFILE
      capact policy apply -f /tmp/inject-storage-policy.yaml

   3. Create Action

      capact act create cap.interface.capactio.capact.validation.action.passing --name test-v4 --type-instances-from-file /tmp/act-input-ti.yaml

      Expected output:

          message: 'Cannot render given action: while rendering Action: while noting output
            artifacts: while resolving backend ID for testUpload: TypeInstance with "injected"
            alias is not a Hub storage (will retry - 0/15)'

5. Missing capact-outputTypeInstances[].backend type.

   1. First change the backend to something in https://github.com/mszostok/os-hub-manifests/blob/policy-syntax/type-instances/manifests/implementation/capactio/capact/validation/action/passing-b.yaml#L73. Next, populate updated manifests.
   2. Update Policy:

      cat > /tmp/inject-storage-policy.yaml << ENDOFFILE
      interface:
        rules:
        - interface:
            path: cap.interface.capactio.capact.validation.action.passing
          oneOf:
          - implementationConstraints:
              attributes:
              - path: cap.attribute.capactio.capact.validation.policy.most-preferred
              requires:
              - path: cap.type.capactio.capact.validation.single-key
            inject:
              requiredTypeInstances:
              - description: Test TypeInstance
                id: ${INJECT_TI}
              - description: Helm backend TypeInstance
                id: ${HELM_STORAGE_TI}
          - implementationConstraints:
              path: cap.implementation.capactio.capact.validation.action.passing-a
        - interface:
            path: cap.*
          oneOf:
          - implementationConstraints: {}
      typeInstance:
        rules: []
      ENDOFFILE
      capact policy apply -f /tmp/inject-storage-policy.yaml

   3. Create Action

      capact act create cap.interface.capactio.capact.validation.action.passing --name test-v5 --type-instances-from-file /tmp/act-input-ti.yaml

      Expected output:

          message: 'Cannot render given action: while rendering Action: while noting output
            artifacts: while resolving backend ID for testUpload: cannot find backend storage
            for specified something alias (will retry - 11/15)'

Related issue(s)

• https://github.com/capactio/capact/issues/624