Add parsing/rego-compilation for x-seal-obligation support

[rchowinfoblox]

Note that I think the obligations should be parsed in the front-end parser, but because the linearization (flattening of the nested contexts) is currently be done in the back-end compiler, the changes for obligations support is all in the back-end side. In a future enhancement, I think the linearization should be done in the front-end parser so that a flattened out AST is available to all back-end compilers.

$ make test
?       github.com/infobloxopen/seal    [no test files]
?       github.com/infobloxopen/seal/cmd        [no test files]
?       github.com/infobloxopen/seal/pkg/ast    [no test files]
?       github.com/infobloxopen/seal/pkg/compiler       [no test files]
?       github.com/infobloxopen/seal/pkg/compiler/error [no test files]
=== RUN   TestCompile
    rego_test.go:180: validate policy: allow subject group foo to manage petstore.pet;
    rego_test.go:181: rego language output generated:

        package foo

        default allow = false
        default deny = false

        base_verbs := {
        }

        allow {
            seal_list_contains(seal_subject.groups, `foo`)
            seal_list_contains(base_verbs[input.type][`manage`], input.verb)
            re_match(`petstore.pet`, input.type)
        }

        obligations := [
        ]

        # rego functions defined by seal

        # Helper to get the token payload.
        seal_subject = payload {
            [header, payload, signature] := io.jwt.decode(input.jwt)
        }

        # seal_list_contains returns true if elem exists in list
        seal_list_contains(list, elem) {
            list[_] = elem
        }
    rego_test.go:180: validate policy: allow subject user foo to manage petstore.pet;
    rego_test.go:181: rego language output generated:

        package foo

        default allow = false
        default deny = false

        base_verbs := {
        }

        allow {
            seal_subject.sub == `foo`
            seal_list_contains(base_verbs[input.type][`manage`], input.verb)
            re_match(`petstore.pet`, input.type)
        }

        obligations := [
        ]

        # rego functions defined by seal

        # Helper to get the token payload.
        seal_subject = payload {
            [header, payload, signature] := io.jwt.decode(input.jwt)
        }

        # seal_list_contains returns true if elem exists in list
        seal_list_contains(list, elem) {
            list[_] = elem
        }
--- PASS: TestCompile (0.00s)
PASS
ok      github.com/infobloxopen/seal/pkg/compiler/rego  (cached)
=== RUN   TestCompiler
--- PASS: TestCompiler (0.00s)
=== RUN   TestLanguages
    compiler_test.go:86: validate list of languages - supported languages: []string{"rego"}
--- PASS: TestLanguages (0.00s)
=== RUN   TestBackend
=== RUN   TestBackend/invalid-nonwildcarded-resource-property-with-subject
=== RUN   TestBackend/multiple-statements
=== RUN   TestBackend/matches
=== RUN   TestBackend/in-operator
=== RUN   TestBackend/obligations-multi-oblig-in-single-stmt
=== RUN   TestBackend/blank-swagger
=== RUN   TestBackend/missing-verb-errors
=== RUN   TestBackend/missing-resource-errors
=== RUN   TestBackend/invalid-nonwildcarded-resource-property
=== RUN   TestBackend/grouping-with-parens
=== RUN   TestBackend/tags
=== RUN   TestBackend/context-2
=== RUN   TestBackend/grouping-with-not-and-parens
=== RUN   TestBackend/context
=== RUN   TestBackend/invalid-resource-format-without-using-family.type-errors
=== RUN   TestBackend/statement-with-not
=== RUN   TestBackend/obligations-simple
=== RUN   TestBackend/obligations-multi-stmt-with-oblig
=== RUN   TestBackend/missing-to-errors
=== RUN   TestBackend/precedence-with-not
=== RUN   TestBackend/blank-subject
=== RUN   TestBackend/no-swagger-actions
=== RUN   TestBackend/support-for-or-operator-simple
=== RUN   TestBackend/support-for-or-operator-context
=== RUN   TestBackend/company.personnel
=== RUN   TestBackend/context-nested
=== RUN   TestBackend/obligations-context
=== RUN   TestBackend/invalid-resource-not-registered
=== RUN   TestBackend/invalid-nonwildcarded-resource-property-in-context
=== RUN   TestBackend/simplest-statement-with-subject
=== RUN   TestBackend/statement-with-and
=== RUN   TestBackend/not-in-operator
=== RUN   TestBackend/obligations-wildcard
--- PASS: TestBackend (0.07s)
    --- PASS: TestBackend/invalid-nonwildcarded-resource-property-with-subject (0.00s)
    --- PASS: TestBackend/multiple-statements (0.00s)
    --- PASS: TestBackend/matches (0.00s)
    --- PASS: TestBackend/in-operator (0.00s)
    --- PASS: TestBackend/obligations-multi-oblig-in-single-stmt (0.00s)
    --- PASS: TestBackend/blank-swagger (0.00s)
    --- PASS: TestBackend/missing-verb-errors (0.00s)
    --- PASS: TestBackend/missing-resource-errors (0.00s)
    --- PASS: TestBackend/invalid-nonwildcarded-resource-property (0.00s)
    --- PASS: TestBackend/grouping-with-parens (0.00s)
    --- PASS: TestBackend/tags (0.00s)
    --- PASS: TestBackend/context-2 (0.00s)
    --- PASS: TestBackend/grouping-with-not-and-parens (0.00s)
    --- PASS: TestBackend/context (0.00s)
    --- PASS: TestBackend/invalid-resource-format-without-using-family.type-errors (0.00s)
    --- PASS: TestBackend/statement-with-not (0.00s)
    --- PASS: TestBackend/obligations-simple (0.00s)
    --- PASS: TestBackend/obligations-multi-stmt-with-oblig (0.00s)
    --- PASS: TestBackend/missing-to-errors (0.00s)
    --- PASS: TestBackend/precedence-with-not (0.00s)
    --- PASS: TestBackend/blank-subject (0.00s)
    --- PASS: TestBackend/no-swagger-actions (0.00s)
    --- PASS: TestBackend/support-for-or-operator-simple (0.00s)
    --- PASS: TestBackend/support-for-or-operator-context (0.00s)
    --- PASS: TestBackend/company.personnel (0.00s)
    --- PASS: TestBackend/context-nested (0.00s)
    --- PASS: TestBackend/obligations-context (0.00s)
    --- PASS: TestBackend/invalid-resource-not-registered (0.00s)
    --- PASS: TestBackend/invalid-nonwildcarded-resource-property-in-context (0.00s)
    --- PASS: TestBackend/simplest-statement-with-subject (0.00s)
    --- PASS: TestBackend/statement-with-and (0.00s)
    --- PASS: TestBackend/not-in-operator (0.00s)
    --- PASS: TestBackend/obligations-wildcard (0.00s)
=== RUN   TestManySwaggers
=== RUN   TestManySwaggers/empty
=== RUN   TestManySwaggers/global-sw1-sw2
=== RUN   TestManySwaggers/global-sw2-sw1
=== RUN   TestManySwaggers/sw1
=== RUN   TestManySwaggers/sw2
=== RUN   TestManySwaggers/global
--- PASS: TestManySwaggers (0.01s)
    --- PASS: TestManySwaggers/empty (0.00s)
    --- PASS: TestManySwaggers/global-sw1-sw2 (0.00s)
    --- PASS: TestManySwaggers/global-sw2-sw1 (0.00s)
    --- PASS: TestManySwaggers/sw1 (0.00s)
    --- PASS: TestManySwaggers/sw2 (0.00s)
    --- PASS: TestManySwaggers/global (0.00s)
PASS
ok      github.com/infobloxopen/seal/pkg/compiler/test  (cached)
=== RUN   TestNextToken
--- PASS: TestNextToken (0.00s)
=== RUN   TestContextToken
--- PASS: TestContextToken (0.00s)
=== RUN   TestNextTokenComment
--- PASS: TestNextTokenComment (0.00s)
PASS
ok      github.com/infobloxopen/seal/pkg/lexer  (cached)
=== RUN   TestWhereClause
=== RUN   TestWhereClause/simple_user
=== RUN   TestWhereClause/simple_group
=== RUN   TestWhereClause/simple_where_clause_compare_equal
=== RUN   TestWhereClause/simple_where_clause_compare_not_equal
=== RUN   TestWhereClause/simple_where_clause_compare_int
=== RUN   TestWhereClause/simple_where_clause_compare_bool
=== RUN   TestWhereClause/single_where_clause_and
=== RUN   TestWhereClause/left_associative_where_clause_and
=== RUN   TestWhereClause/where_clause_grouped_conditions
=== RUN   TestWhereClause/where_clause_multiple_grouped_conditions
--- PASS: TestWhereClause (0.00s)
    --- PASS: TestWhereClause/simple_user (0.00s)
    --- PASS: TestWhereClause/simple_group (0.00s)
    --- PASS: TestWhereClause/simple_where_clause_compare_equal (0.00s)
    --- PASS: TestWhereClause/simple_where_clause_compare_not_equal (0.00s)
    --- PASS: TestWhereClause/simple_where_clause_compare_int (0.00s)
    --- PASS: TestWhereClause/simple_where_clause_compare_bool (0.00s)
    --- PASS: TestWhereClause/single_where_clause_and (0.00s)
    --- PASS: TestWhereClause/left_associative_where_clause_and (0.00s)
    --- PASS: TestWhereClause/where_clause_grouped_conditions (0.00s)
    --- PASS: TestWhereClause/where_clause_multiple_grouped_conditions (0.00s)
=== RUN   TestLetStatements
--- PASS: TestLetStatements (0.00s)
PASS
ok      github.com/infobloxopen/seal/pkg/parser (cached)
=== RUN   TestLookupOperator
=== RUN   TestLookupOperator/invalid_empty
=== RUN   TestLookupOperator/equal_to
=== RUN   TestLookupOperator/not_equal
=== RUN   TestLookupOperator/less_than
=== RUN   TestLookupOperator/greater_than
=== RUN   TestLookupOperator/less_than_or_equal_to
=== RUN   TestLookupOperator/greater_than_or_equal_to
=== RUN   TestLookupOperator/not
=== RUN   TestLookupOperator/and
=== RUN   TestLookupOperator/or
=== RUN   TestLookupOperator/matches
--- PASS: TestLookupOperator (0.00s)
    --- PASS: TestLookupOperator/invalid_empty (0.00s)
    --- PASS: TestLookupOperator/equal_to (0.00s)
    --- PASS: TestLookupOperator/not_equal (0.00s)
    --- PASS: TestLookupOperator/less_than (0.00s)
    --- PASS: TestLookupOperator/greater_than (0.00s)
    --- PASS: TestLookupOperator/less_than_or_equal_to (0.00s)
    --- PASS: TestLookupOperator/greater_than_or_equal_to (0.00s)
    --- PASS: TestLookupOperator/not (0.00s)
    --- PASS: TestLookupOperator/and (0.00s)
    --- PASS: TestLookupOperator/or (0.00s)
    --- PASS: TestLookupOperator/matches (0.00s)
PASS
ok      github.com/infobloxopen/seal/pkg/token  (cached)
=== RUN   TestIsNilInterface
--- PASS: TestIsNilInterface (0.00s)
=== RUN   TestNewTypeFromOpenAPIv3
    types_test.go:17: got type: petstore.pet
    types_test.go:19: got action: &types.swaggerAction{name:"allow", schema:(*openapi3.SchemaRef)(0xc00000efa0)}
    types_test.go:23:     TODO: get type schema for action
    types_test.go:19: got action: &types.swaggerAction{name:"deny", schema:(*openapi3.SchemaRef)(nil)}
    types_test.go:23:     TODO: get type schema for action
--- PASS: TestNewTypeFromOpenAPIv3 (0.00s)
PASS
ok      github.com/infobloxopen/seal/pkg/types  (cached)

$ make demo
./seal compile \
        -s docs/source/examples/petstore/petstore.jwt.swagger \
        -s docs/source/examples/petstore/petstore.tags.swagger \
        -s docs/source/examples/petstore/petstore.all.swagger \
        -f docs/source/examples/petstore/petstore.all.seal \
        > docs/source/examples/petstore/petstore.all.rego.compiled
INFO[0000] logging level                                 logging.level=info
cat docs/source/examples/petstore/petstore.all.rego.compiled

package petstore.all

default allow = false
default deny = false

base_verbs := {
    "petstore.order": {
        "approve": [
            "approve",
        ],
        "deliver": [
            "deliver",
        ],
        "inspect": [
            "list",
            "watch",
        ],
        "manage": [
            "create",
            "delete",
            "update",
            "get",
            "list",
            "watch",
        ],
        "read": [
            "get",
            "list",
            "watch",
        ],
        "ship": [
            "ship",
        ],
        "use": [
            "update",
            "get",
            "list",
            "watch",
        ],
    },
    "petstore.pet": {
        "buy": [
            "buy",
        ],
        "inspect": [
            "list",
            "watch",
        ],
        "manage": [
            "create",
            "delete",
            "update",
            "get",
            "list",
            "watch",
        ],
        "provision": [
            "provision",
        ],
        "read": [
            "get",
            "list",
            "watch",
        ],
        "sell": [
            "sell",
        ],
        "use": [
            "update",
            "get",
            "list",
            "watch",
        ],
    },
    "petstore.user": {
        "inspect": [
            "list",
            "watch",
        ],
        "manage": [
            "create",
            "delete",
            "update",
            "get",
            "list",
            "watch",
        ],
        "read": [
            "get",
            "list",
            "watch",
        ],
        "sign_in": [
            "sign_in",
        ],
        "use": [
            "update",
            "get",
            "list",
            "watch",
        ],
    },
}

deny {
    seal_list_contains(base_verbs[input.type][`deliver`], input.verb)
    re_match(`petstore.order`, input.type)
    seal_list_contains(seal_subject.groups, `boss`)
}

deny {
    seal_list_contains(base_verbs[input.type][`use`], input.verb)
    re_match(`petstore.order`, input.type)

    some i
    input.ctx[i]["id"] == "-1"
}

deny {
    seal_list_contains(base_verbs[input.type][`use`], input.verb)
    re_match(`petstore.user`, input.type)

    some i
    input.ctx[i]["id"] == "-1"
}

deny {
    seal_list_contains(base_verbs[input.type][`use`], input.verb)
    re_match(`petstore.order`, input.type)
    seal_subject.iss != "context.petstore.swagger.io"
}

deny {
    seal_list_contains(base_verbs[input.type][`use`], input.verb)
    re_match(`petstore.user`, input.type)
    seal_subject.iss != "context.petstore.swagger.io"
}

deny {
    seal_list_contains(base_verbs[input.type][`deliver`], input.verb)
    re_match(`petstore.order`, input.type)

    some i
    input.ctx[i]["status"] == "delivered"
}

deny {
    seal_list_contains(seal_subject.groups, `regexp`)
    seal_list_contains(base_verbs[input.type][`use`], input.verb)
    re_match(`petstore.*`, input.type)
    re_match(`@petstore.swagger.io$`, seal_subject.jti)
}

deny {
    seal_list_contains(seal_subject.groups, `everyone`)
    seal_list_contains(base_verbs[input.type][`use`], input.verb)
    re_match(`petstore.*`, input.type)
    seal_subject.iss != "petstore.swagger.io"
}

deny {
    seal_list_contains(seal_subject.groups, `everyone`)
    seal_list_contains(base_verbs[input.type][`buy`], input.verb)
    re_match(`petstore.pet`, input.type)

    some i
    input.ctx[i]["age"] <= 2
    input.ctx[i]["name"] == "specificPetName"
}

deny {
    seal_list_contains(seal_subject.groups, `banned`)
    seal_list_contains(base_verbs[input.type][`manage`], input.verb)
    re_match(`petstore.*`, input.type)
}

deny {
    seal_list_contains(seal_subject.groups, `managers`)
    seal_list_contains(base_verbs[input.type][`sell`], input.verb)
    re_match(`petstore.pet`, input.type)

    some i
    input.ctx[i]["status"] != "available"
}

deny {
    seal_list_contains(seal_subject.groups, `fussy`)
    seal_list_contains(base_verbs[input.type][`buy`], input.verb)
    re_match(`petstore.pet`, input.type)
    not line13_not1_cnd
}

line13_not1_cnd {
    some i
    input.ctx[i]["neutered"]

    not line13_not2_cnd
}

line13_not2_cnd {
    some i
    input.ctx[i]["potty_trained"]
}

allow {
    seal_list_contains(seal_subject.groups, `fussy`)
    seal_list_contains(base_verbs[input.type][`buy`], input.verb)
    re_match(`petstore.pet`, input.type)
    not line14_not1_cnd
}

line14_not1_cnd {
    some i
    input.ctx[i]["neutered"]
    input.ctx[i]["potty_trained"]
}

deny {
    seal_list_contains(seal_subject.groups, `everyone`)
    seal_list_contains(base_verbs[input.type][`buy`], input.verb)
    re_match(`petstore.pet`, input.type)

    some i
    input.ctx[i]["tags"]["endangered"] == "true"
}

allow {
    seal_list_contains(seal_subject.groups, `operators`)
    seal_list_contains(base_verbs[input.type][`use`], input.verb)
    re_match(`petstore.*`, input.type)
}

allow {
    seal_list_contains(seal_subject.groups, `managers`)
    seal_list_contains(base_verbs[input.type][`manage`], input.verb)
    re_match(`petstore.*`, input.type)
}

allow {
    seal_subject.sub == `cto@petstore.swagger.io`
    seal_list_contains(base_verbs[input.type][`manage`], input.verb)
    re_match(`petstore.*`, input.type)
}

allow {
    seal_list_contains(seal_subject.groups, `everyone`)
    seal_list_contains(base_verbs[input.type][`inspect`], input.verb)
    re_match(`petstore.pet`, input.type)
}

allow {
    seal_list_contains(seal_subject.groups, `customers`)
    seal_list_contains(base_verbs[input.type][`read`], input.verb)
    re_match(`petstore.pet`, input.type)
}

allow {
    seal_list_contains(seal_subject.groups, `customers`)
    seal_list_contains(base_verbs[input.type][`buy`], input.verb)
    re_match(`petstore.pet`, input.type)

    some i
    input.ctx[i]["status"] == "available"
}

allow {
    seal_list_contains(seal_subject.groups, `breeders_maltese`)
    seal_list_contains(base_verbs[input.type][`buy`], input.verb)
    re_match(`petstore.pet`, input.type)

    some i
    input.ctx[i]["status"] == "reserved"
    input.ctx[i]["breed"] == "maltese"
}

allow {
    seal_list_contains(seal_subject.groups, `employees`)
    seal_list_contains(base_verbs[input.type][`inspect`], input.verb)
    re_match(`petstore.order`, input.type)

    some i
    input.ctx[i]["status"] == "delivered"
}

allow {
    seal_list_contains(seal_subject.groups, `supervisors`)
    seal_list_contains(base_verbs[input.type][`manage`], input.verb)
    re_match(`petstore.user`, input.type)

    some i
    re_match(`.*@acme.com`, input.ctx[i]["email"])
}

obligations := [
    `(ctx.marketplace != "amazon")`,
    `(ctx.occupation != "unemployed")`,
    `(ctx.salary > 200000)`,
]

# rego functions defined by seal

# Helper to get the token payload.
seal_subject = payload {
    [header, payload, signature] := io.jwt.decode(input.jwt)
}

# seal_list_contains returns true if elem exists in list
seal_list_contains(list, elem) {
    list[_] = elem
}

cp docs/source/examples/petstore/petstore.all.rego.compiled docs/source/examples/petstore/petstore.all.rego
# beware that check-rego.sh reformats the compiled rego files...
./docs/source/examples/check-rego.sh docs/source/examples/petstore
+ IMAGE=openpolicyagent/opa:latest
+ [[ -z docs/source/examples/petstore ]]
+ [[ ! -d docs/source/examples/petstore ]]
++ cd docs/source/examples/petstore
++ /bin/pwd
+ TOP=/home/riccho/go/src/github.com/infobloxopen/seal/docs/source/examples/petstore
+ cd /home/riccho/go/src/github.com/infobloxopen/seal/docs/source/examples/petstore
+ docker run -v /home/riccho/go/src/github.com/infobloxopen/seal/docs/source/examples/petstore:/data -w /data openpolicyagent/opa:latest fmt -w .
+ case "$(basename $0)" in
++ basename ./docs/source/examples/check-rego.sh
+ docker run --rm -v /home/riccho/go/src/github.com/infobloxopen/seal/docs/source/examples/petstore:/data -w /data openpolicyagent/opa:latest test -v petstore.all.rego petstore.all.test_bench.rego petstore.all.test.rego petstore.all.mock.json
data.petstore.all.test_in: PASS (1.503357ms)
data.petstore.all.test_in_negative: PASS (954.601µs)
data.petstore.all.test_regexp: PASS (29.436824ms)
data.petstore.all.test_regexp_negative: PASS (881.991µs)
data.petstore.all.test_ctx_usage_multiply: PASS (1.019189ms)
data.petstore.all.test_ctx_usage_negative_multi_ctx: PASS (914.75µs)
data.petstore.all.test_ctx_usage_negative: PASS (919.64µs)
data.petstore.all.test_use_tags: PASS (2.337777ms)
data.petstore.all.test_use_tags_negative: PASS (1.845155ms)
data.petstore.all.test_use_tags_negative_missing_endangered_tag: PASS (1.925184ms)
data.petstore.all.test_use_petstore_jwt: PASS (2.147398ms)
data.petstore.all.test_use_petstore_jwt_negative: PASS (1.707941ms)
data.petstore.all.test_banned_deny: PASS (972.64µs)
data.petstore.all.test_banned_deny_negative: PASS (800.772µs)
data.petstore.all.test_inspect: PASS (700.834µs)
data.petstore.all.test_inspect_negative: PASS (1.494422ms)
data.petstore.all.test_read: PASS (798.515µs)
data.petstore.all.test_read_negative: PASS (1.072165ms)
data.petstore.all.test_manage_cto: PASS (765.31µs)
data.petstore.all.test_blank_subject: PASS (977.926µs)
data.petstore.all.test_bench_deny_in_map_species: PASS (164.15µs)
data.petstore.all.test_bench_deny_in_map_species_2nd: PASS (152.485µs)
data.petstore.all.test_bench_deny_in_map_species_negative: PASS (136.155µs)
data.petstore.all.test_bench_deny_in_map_species_negative_2nd: PASS (133.433µs)
--------------------------------------------------------------------------------
PASS: 24/24
+ git diff --exit-code petstore.all.rego petstore.all.test_bench.rego petstore.all.test.rego
git diff --exit-code docs/source/examples/petstore
### petstore example passed REGO OPA tests

[rchowinfoblox]

Submitted for initial review, existing unit-tests haven't been fixed yet, and new unit-tests need to be added.