Follow symlinks when building bundles

[bnordbo]

This issue is effectively a duplicate of #4335, but contains additional use-cases for symlinks when building bundles. For the record, this is tested with OPA 0.58 .0 on Darwin, Go 1.24.1.

The problem is that opa build -b ignores symlinks:

$ echo -e "package foo\none = 1" > ../foo.rego
$ ln -s ../foo.rego
$ opa build -b .
$ tar zft bundle.tar.gz
/data.json
$

Some build systems, in particular Bazel, uses symlinks extensively to create a sandbox before building. We tried making rules for building OPA bundles in order to depend on these from other build targets. Alas this doesn't work as Bazel will always build a sanbox with symlinks to the actual rego files, and run build commands from within.

The ideal solution would be a way to specify that opa build should follow symlinks, also when -b is present. There is ample precedence for such command line switces, e.g. --follow-symkilns in GNU sed.

Altrenatively, while not a solution, this constraint does not seem to be documented. It probably should be, as the current behaviour can be confusing. It would also be nice if there was a warning for ignored files.

[bnordbo]

Another use-case for this that I just came across is being able to load rules from Kubernetes configmaps. It turns out that these are actually mounted in ..YYYY_mm_DD_HH_MM_SS.ns inside the mount-point, and then files are symlinked individually. This causes opa run -b to fail with a somewhat misleading error message (as it ignores .manifest):

$ opa run -b --ignore='..*' rego_lib.bundle.tar.gz rego
error: initialization error: detected overlapping roots in bundle manifest with: [rego rego_lib.bundle.tar.gz]

(--ignore is just to prevent OPA from failing by trying to load a directory with an invalid structure)

[ashutosh-narkar]

Adding a new flag to opa build to follow symlinks seems fine. This would require changes in the bundle reader. In the meantime if you'd like to improve the docs with added info about symlinks feel free to contribute. Thanks.

[ffortier]

As a workaround, I've also made my own bazel rules for opa that seems to be working: https://github.com/ticketmaster/rules_opa.

[tjons]

taking this one per @charlieegan3's recommendation - @charlieegan3 or @ashutosh-narkar could one of you assign to me? easier to remember it when I get busy that way :)

[stale[bot]]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days. Although currently inactive, the issue could still be considered and actively worked on in the future. More details about the use-case this issue attempts to address, the value provided by completing it or possible solutions to resolve it would help to prioritize the issue.

[tjons]

Not stale. Should be done this weekend.

[stale[bot]]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days. Although currently inactive, the issue could still be considered and actively worked on in the future. More details about the use-case this issue attempts to address, the value provided by completing it or possible solutions to resolve it would help to prioritize the issue.

[tjons]

PR for this here: https://github.com/open-policy-agent/opa/pull/6800

[stale[bot]]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days. Although currently inactive, the issue could still be considered and actively worked on in the future. More details about the use-case this issue attempts to address, the value provided by completing it or possible solutions to resolve it would help to prioritize the issue.