Closed jakubdyszkiewicz closed 1 year ago
@jakubdyszkiewicz thanks for reporting this. Are you using bundles to serve the policies or some other way? I tried with your policy to repro this.
policy.rego
package envoy.authz
import input.attributes.request.http as http_request
default allow = false
allow {
http_request.method == "GET"
}
config.yaml
plugins:
envoy_ext_authz_grpc:
addr: :9191
dry-run: false
enable-reflection: true
decision_logs:
console: true
Then run opa-envoy:
$ opa_envoy run --server --config-file=config.yaml policy.rego
{"addrs":[":8181"],"diagnostic-addrs":[],"level":"info","msg":"Initializing server.","time":"2023-01-25T10:55:12-08:00"}
{"level":"info","msg":"Starting decision logger.","plugin":"decision_logs","time":"2023-01-25T10:55:12-08:00"}
{"addr":":9191","dry-run":false,"enable-reflection":true,"level":"info","msg":"Starting gRPC server.","path":"test/hello","query":"","time":"2023-01-25T10:55:12-08:00"}
^C{"level":"info","msg":"Shutting down...","time":"2023-01-25T10:55:14-08:00"}
{"level":"info","msg":"Server shutdown.","time":"2023-01-25T10:55:14-08:00"}
{"level":"info","msg":"Stopping decision logger.","plugin":"decision_logs","time":"2023-01-25T10:55:14-08:00"}
The bug is only visible with multiple rego policies. If you change const policies = 2
to = 1
in my repro repo and it works. Additionally, just starting OPA with two policies does not expose the problem, the bug is only visible when you send requests.
We do not use bundles, we exchange rego policies between the control plane and data plane via our own protocol (xDS-like). Then we push them to the store like in the example repo using rt.Store.UpsertPolicy
.
Can you help me understand why you add the same policy to the store twice under a different id? If you compile the modules, it would generate an error. For ex.
const policy = `
package envoy.authz
import input.attributes.request.http as http_request
default allow = false
allow {
http_request.method == "GET"
}`
_, err = ast.CompileModules(map[string]string{
"example1.rego": policy,
"example2.rego": policy,
})
You'll see an error like rego_type_error: multiple default rules data.envoy.authz.allow found
. Now the panic you reported is something that needs more investigation. I would have expected OPA to maybe give an error and not crash. If you need more info about integrating with OPA's API, you can find that here.
The same policy was just a simplification. The real use case would be something like this
1) Mesh operator defines a global policy that denies all requests with specific URL/header (i.e. jndi
to protect from Log4Shell)
2) Service owner defines a policy for their own service that allows only GET requests.
After reading more, I see this was a missing part for me https://www.openpolicyagent.org/docs/latest/faq/#collaboration-using-import
I was under the impression that policies would be executed separately. My mistake!
I see that to implement such a use case we need to have a base policy from the mesh operator
package base
default allow = true
deny {
input.parsed_path == ["jndi"]
}
a policy from the service owner
package service
import input.attributes.request.http as http_request
default allow = false
allow {
http_request.method == "GET"
}
a policy to stitch this together
package kmesh
import data.service
import data.base
allow {
service.allow
base.allow
not service.deny
not base.deny
}
and configure OPA Envoy plugin plugins.envoy_ext_authz_grpc.path=kmesh/allow
This works 🎉
I think the issue still might be valid. It would be great if the agent returned a better error.
@jakubdyszkiewicz thanks for confirming. We will into the error message separately.
Hey 👋
Short description
We use OPA in Kong Mesh by embedding it in our sidecar. We noticed that whenever we add multiple policies to the runtime store and send a request, OPA agent panics. I built a minimal repo https://github.com/jakubdyszkiewicz/opa-envoy-panic-repro to reproduce it
If we somehow misuse the API, I'd love to know what we should improve.
Here is the log.
I tried it with both 0.48.0 and 0.47.4, same result.
Steps To Reproduce
I built a minimal repo https://github.com/jakubdyszkiewicz/opa-envoy-panic-repro
Expected behavior
No panic.
Thanks for all the help!