search

open-policy-agent / opa

Open Policy Agent (OPA) is an open source, general-purpose policy engine.
https://www.openpolicyagent.org
Apache License 2.0
9.76k stars 1.36k forks source link

Add binary testing for arm64 platform via Github Actions #4591

Closed odidev closed 2 years ago

odidev commented 2 years ago

Hi Team,

I am working with open-policy-agent/opa on the Linux/ARM64 platform.

We know that open-policy-agent/opa already supports Linux/ARM64 platform, and releasing the ARM64 binaries as well using the cross-compilation.

But I can see in Github Actions that the AMD64 binaries are also tested before releasing.

May I know, are you also planning to include the testing of Linux/ARM64 binaries before releasing, via Github Actions, in the future?

If interested there are machines hosted through the works on arm (https://github.com/WorksOnArm), for arm64 platform.

Please share your opinion on the same. If required, I will be happy to contribute.

srenatus commented 2 years ago

Better test coverage for Linux/arm64 (and macos/arm64) would be desirable! I've tried some stuff using the binfmt ext stuff earlier, but hit a blocker that led us to only provide Linux/arm64 docker images with our static binary ... https://github.com/bytecodealliance/wasmtime-go/issues/119

How does worksonarm work? Are there publicly accessible workers?

srenatus commented 2 years ago

Btw -- I was brief because on mobile for the other message -- any improvements to CI that increase our confidence in the arm64 builds, or just bring them on par with the amd64 builds, are very welcome. 👍

jsrz commented 2 years ago

Hi @srenatus,

Works on Arm is an initiative that provides free access to Arm based clusters (on Equinix). This is for the purpose of giving open source projects access to Arm HW to support build-test loops. To apply for access, you just have to open an issue with the project (linked below). Given this is a CNCF project, I would imagine it would get accepted.

https://github.com/WorksOnArm/equinix-metal-arm64-cluster

Keep in mind, this is for access to Arm based HW. The actual maintenance of the build systems deployed on the Arm cluster would still need to be done by the project.

If you decide to apply, let us know so we can push it along (I'm an Arm employee by the way, which is separate from Equinix).

odidev commented 2 years ago

Hi @srenatus

I’m testing open policy agent on arm64 platform on kubernetes using this documentation. But I’m failing on this step because openpolicyagent/opa:0.39.0-rootless docker image is not available for arm64 platform. I have tried to build this by creating a opa_linux_arm64 binary for arm64 platform by using make ci-build-linux command. And used this image to create docker image using docker build -t odidev/opa-rootles:rootless --build-arg USER=1000:1000 --build-arg BASE=gcr.io/distroless/cc --build-arg BIN_DIR=_release/0.40.0-dev . .

But when I'm using this image, opa pods are crashing with below logs. 

xyz@arm38:~$ kubectl get pods -n opa 
NAME                  READY   STATUS             RESTARTS         AGE 
opa-d499fd877-2s276   1/2     CrashLoopBackOff   72 (4m28s ago)   5h48m 
xyz@arm38:~$ kubectl describe pod opa-d499fd877-2s276 -n opa 
Name:         opa-d499fd877-2s276 
Namespace:    opa 
Priority:     0 
Node:         minikube/192.168.49.2 
Start Time:   Tue, 26 Apr 2022 07:59:44 +0000 
Labels:       app=opa 
              pod-template-hash=d499fd877 
Annotations:  <none> 
Status:       Running 
IP:           172.17.0.4 
IPs: 
  IP:           172.17.0.4 
Controlled By:  ReplicaSet/opa-d499fd877 
Containers: 
  opa: 
    Container ID:  docker://416b9e3addeb261c020dcd992a56a7192881649d72ef33fd7817efaacdde63c3 
    Image:         odidev/opa-rootles:rootless 
    Image ID:      docker-pullable://odidev/opa-rootles@sha256:542edd919e0cfbe25781a5a608f0afe7b5723246c16bf70c22a960051514cc58 
    Port:          <none> 
    Host Port:     <none> 
    Args: 
      run 
      --server 
      --tls-cert-file=/certs/tls.crt 
      --tls-private-key-file=/certs/tls.key 
      --addr=0.0.0.0:8443 
      --addr=http://127.0.0.1:8181 
      --set=services.default.url=http://host.minikube.internal:8888 
      --set=bundles.default.resource=bundle.tar.gz 
      --log-format=json-pretty 
      --set=status.console=true 
      --set=decision_logs.console=true 
    State:          Running 
      Started:      Tue, 26 Apr 2022 08:00:03 +0000 
    Ready:          True 
    Restart Count:  0 
    Liveness:       http-get https://:8443/health delay=3s timeout=1s period=5s #success=1 #failure=3 
    Readiness:      http-get https://:8443/health%3Fplugins&bundle delay=3s timeout=1s period=5s #success=1 #failure=3 
    Environment:    <none> 
    Mounts: 
      /certs from opa-server (ro) 
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-bchbd (ro) 
  kube-mgmt: 
    Container ID:  docker://e4199464588214e007af45356e28a4e5024b2e95a232a159cc345ddfd6a5b5b5 
    Image:         openpolicyagent/kube-mgmt:2.0.1 
    Image ID:      docker-pullable://openpolicyagent/kube-mgmt@sha256:d3f0e7011e92864a5fcbe2fd1e0310ec860a3fc05bbe9bad4aaa4a230f9c73af 
    Port:          <none> 
    Host Port:     <none> 
    Args: 
      --replicate-cluster=v1/namespaces 
      --replicate=networking.k8s.io/v1/ingresses 
    State:          Waiting 
      Reason:       CrashLoopBackOff 
    Last State:     Terminated 
      Reason:       Error 
      Exit Code:    1 
      Started:      Tue, 26 Apr 2022 13:49:16 +0000 
      Finished:     Tue, 26 Apr 2022 13:49:16 +0000 
    Ready:          False 
    Restart Count:  73 
    Environment:    <none> 
    Mounts: 
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-bchbd (ro) 
Conditions: 
  Type              Status 
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes: 
  opa-server: 
    Type:        Secret (a volume populated by a Secret) 
    SecretName:  opa-server 
    Optional:    false 
  kube-api-access-bchbd: 
    Type:                    Projected (a volume that contains injected data from multiple sources) 
    TokenExpirationSeconds:  3607 
    ConfigMapName:           kube-root-ca.crt 
    ConfigMapOptional:       <nil> 
    DownwardAPI:             true 
QoS Class:                   BestEffort 
Node-Selectors:              <none> 
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s 
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s 
Events: 
  Type     Reason   Age                     From     Message 
  ----     ------   ----                    ----     ------- 
  Warning  BackOff  16s (x1623 over 5h49m)  kubelet  Back-off restarting failed container

Can you please provide us with some pointers on this problem?

srenatus commented 2 years ago

Hmm those logs don't contain anything helpful, do they?

As you've noticed, we don't publish the combination of rootless + static; static being the only thing we build for linux/arm64. I think the quickest way forward is to build your own rootless container using the static binary, something along the lines of

FROM gcr.io/distroless/cc
USER 1000:1000

# TARGETOS and TARGETARCH are automatic platform args injected by BuildKit
# https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope
ARG TARGETOS
ARG TARGETARCH

COPY --from=openpolicyagent/opa:latest-static /opa /opa

ENTRYPOINT ["/opa"]
CMD ["run"]

where we're basically snatching the static arm binary from the published openpolicyagent/opa:latest-static image... ⚠️ I haven't tried this, and there may be details to take care of to ensure that the binary is taken from the linux/arm64 variant... LMK if you get stuck or unstuck here, please 😅

srenatus commented 2 years ago

To build the binary yourself, something along the lines of what happens in CI is probably the way to go: https://github.com/open-policy-agent/opa/blob/main/.github/workflows/post-merge.yaml#L95-L100 👇 

env GOARCH=arm64 make ci-go-ci-build-linux-static
anderseknert commented 2 years ago

Just to add to that: all images (including -static) will be rootless by default later this year.

odidev commented 2 years ago

 I haven't tried this, and there may be details to take care of to ensure that the binary is taken from the linux/arm64 variant... LMK if you get stuck or unstuck here, please 

I have tried this dockerfile that you have shared. It is successfully working for me on arm64 platform.

Just to add to that: all images (including -static) will be rootless by default later this year.

Can you please let me know that for future release of docker image which binary are you going to use static binary or non-static binary? Please let me know I’m ready to help.

anderseknert commented 2 years ago

I think we'll continue to have both the static and the non-static images around for the near future. I don't think the Wasm stuff works in the non-static context still, but perhaps @srenatus knows if this is about to change.

srenatus commented 2 years ago

I have tried this dockerfile that you have shared. It is successfully working for me on arm64 platform.

Yay. So I guess it comes down to this: Is the workaround too cumbersome to be acceptable until all images switch to rootless?

Can you please let me know that for future release of docker image which binary are you going to use static binary or non-static binary? Please let me know I’m ready to help.

We'll probably keep static and "default" (non-static, dynamic build). The latter is able to run wasm modules, the former is not. The only planned change is to collapse the "rootless" variant with the "default" variants, i.e. go from

to

and having them all be rootless.

The reason we're not having "default" (non-static) rootless images for linux/arm64 is that we couldn't properly test them (i.e. basically this issue). Using the binfmt_ext emulation led to https://github.com/bytecodealliance/wasmtime-go/issues/119.

So, if we had sorted out a way to properly test the linux/arm64 binaries (or run the golang tests for that matter) on linux/arm64 including the pieces that need wasm libs, we'd be able to build the default (and default-rootless) variants for linux/arm64.

At this point, however, we haven't planned to take on the work required to set up a self-hosted GHA runner on the equinix-hosted hardware, even if they accepted our proposal. I don't think this is going to be a onetime thing, these runners will need to be maintained, and cared for... 🤔

jsrz commented 2 years ago

@srenatus, just to be clear,works on Arm would allow for use to run and maintain the runners for more than just a one time thing.

srenatus commented 2 years ago

@jsrz sorry if my statement wasn't worded properly -- I think I understand that the works-on-arm stuff would give us shell access to some Arm hardware, and it would be our task to set that up as a github actions runner. By "not a one time thing" I was referring to the work involved in maintaining that setup. That won't be a do-and-forget kind of thing, there are going to be updates to do, etc. Am I mistaken? 😃

jsrz commented 2 years ago

@srenatus got, I understand. You are not mistaken, it would certainly need maintenance.

stale[bot] commented 2 years ago

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.

anderseknert commented 2 years ago

I don't think this is likely to be something we'll add anytime soon, so let's revisit this in the future if we feel it's needed. Closing for now.

odidev commented 2 years ago

Hi Team,

I have successfully tested the opa binary for arm64 platform using opa-wasm-builder docker image. I have created opa-wasm-builder docker image using this README.md file for arm64 platform. Using this image, I have successfully built the opa and opa-static binaries. Also, I successfully tested both the binaries for the arm64 platform. I have used below mentioned command to test the build and test the binary: - 

# sudo make wasm-test
# sudo make ci-build-linux ci-build-linux-static
# sudo make ci-binary-smoke-test-rego BINARY=opa_linux_arm64
# sudo make ci-binary-smoke-test-wasm BINARY=opa_linux_arm64
# sudo make ci-binary-smoke-test-rego BINARY=opa_linux_arm64_static

The reason we're not having "default" (non-static) rootless images for linux/arm64 is that we couldn't properly test them (i.e. basically this issue). Using the binfmt_ext emulation led to bytecodealliance/wasmtime-go#119.

May I know which test you are running which leads you to this error?

srenatus commented 2 years ago

May I know which test you are running which leads you to this error?

Sure! I think it's been the wasm e2e tests: https://github.com/open-policy-agent/opa/blob/c54ddd31d7f4e8d84d8f9aa8f7e907245c1f8c3f/Makefile#L144

srenatus commented 2 years ago

For completeness' sake, most our tests are Golang unit tests: the binary smoke tests only assert that the binary does something; as opposed to failing to find some lib or such a thing. We'd need to pass make test on a platform, too.

odidev commented 2 years ago

Hi @srenatus

I have run the test that you have suggested and both of the tests successfully passed on the arm64 platform using sudo make wasm-sdk-e2e-test and sudo make test command. I have also attached the logs. 

# sudo make wasm-sdk-e2e-test

--------------------------- Huge number of logs-----------------------------------
    --- PASS: TestWasmE2E/test/cases/testdata/withkeyword/test-withkeyword-1025.yaml/withkeyword/with_stack_overwrites_(data) (0.36s)
    --- PASS: TestWasmE2E/test/cases/testdata/withkeyword/test-withkeyword-1018.yaml/withkeyword/with_vars (0.40s)
    --- PASS: TestWasmE2E/test/cases/testdata/withkeyword/test-withkeyword-1041.yaml/withkeyword/with_virtual_doc_any_index (0.32s)
    --- PASS: TestWasmE2E/test/cases/testdata/withkeyword/test-withkeyword-1040.yaml/withkeyword/with_virtual_doc_exact_value (0.34s)
    --- PASS: TestWasmE2E/test/cases/testdata/withkeyword/test-withkeyword-1043.yaml/withkeyword/with_virtual_doc_not_specific_index (0.38s)
    --- PASS: TestWasmE2E/test/cases/testdata/withkeyword/test-withkeyword-1042.yaml/withkeyword/with_virtual_doc_specific_index (0.34s)
PASS
ok      github.com/open-policy-agent/opa/internal/wasm/sdk/test/e2e     503.586s

subham@ip-172-31-46-24:~/opa$ sudo make test
make[1]: Entering directory '/home/subham/opa/wasm'
make: '_obj/opa.wasm' is up to date.
make: '_obj/callgraph.csv' is up to date.
make[1]: Leaving directory '/home/subham/opa/wasm'
cp wasm/_obj/opa.wasm internal/compiler/wasm/opa/opa.wasm
cp wasm/_obj/callgraph.csv internal/compiler/wasm/opa/callgraph.csv
CGO_ENABLED=1 GOFLAGS="-buildmode=exe" go generate
2022/06/20 08:15:14 WARN: not categorized: assign
2022/06/20 08:15:14 WARN: not categorized: cast_array
2022/06/20 08:15:14 WARN: not categorized: cast_boolean
2022/06/20 08:15:14 WARN: not categorized: cast_null
2022/06/20 08:15:14 WARN: not categorized: cast_object
2022/06/20 08:15:14 WARN: not categorized: cast_set
2022/06/20 08:15:14 WARN: not categorized: cast_string
2022/06/20 08:15:14 WARN: not categorized: eq
2022/06/20 08:15:14 WARN: not categorized: print
2022/06/20 08:15:14 WARN: not categorized: re_match
2022/06/20 08:15:14 WARN: not categorized: set_diff
2022/06/20 08:15:14 WARN: not categorized: type_name
CGO_ENABLED=1 GOFLAGS="-buildmode=exe" go test -tags=opa_wasm,slow ./...
?       github.com/open-policy-agent/opa        [no test files]
ok      github.com/open-policy-agent/opa/ast    3.912s
ok      github.com/open-policy-agent/opa/ast/internal/scanner   0.017s
?       github.com/open-policy-agent/opa/ast/internal/tokens    [no test files]
ok      github.com/open-policy-agent/opa/ast/location   0.015s
?       github.com/open-policy-agent/opa/build/generate-cli-docs        [no test files]
?       github.com/open-policy-agent/opa/build/generate-man     [no test files]
ok      github.com/open-policy-agent/opa/bundle 0.174s
ok      github.com/open-policy-agent/opa/capabilities   0.341s
ok      github.com/open-policy-agent/opa/cmd    7.665s
?       github.com/open-policy-agent/opa/cmd/internal/exec      [no test files]
ok      github.com/open-policy-agent/opa/compile        0.296s
ok      github.com/open-policy-agent/opa/config 0.021s
ok      github.com/open-policy-agent/opa/cover  0.014s
ok      github.com/open-policy-agent/opa/dependencies   0.075s
ok      github.com/open-policy-agent/opa/download       31.185s
?       github.com/open-policy-agent/opa/features/tracing       [no test files]
?       github.com/open-policy-agent/opa/features/wasm  [no test files]
ok      github.com/open-policy-agent/opa/format 0.058s
?       github.com/open-policy-agent/opa/internal/bundle        [no test files]
ok      github.com/open-policy-agent/opa/internal/bundle/inspect        0.012s
?       github.com/open-policy-agent/opa/internal/cidr/merge    [no test files]
?       github.com/open-policy-agent/opa/internal/cmd/genbuiltinmetadata        [no test files]
?       github.com/open-policy-agent/opa/internal/cmd/genopacapabilities        [no test files]
?       github.com/open-policy-agent/opa/internal/cmd/genopawasm        [no test files]
ok      github.com/open-policy-agent/opa/internal/compiler/wasm 0.047s
?       github.com/open-policy-agent/opa/internal/compiler/wasm/opa     [no test files]
ok      github.com/open-policy-agent/opa/internal/config        0.017s
?       github.com/open-policy-agent/opa/internal/debug [no test files]
ok      github.com/open-policy-agent/opa/internal/deepcopy      0.004s
?       github.com/open-policy-agent/opa/internal/distributedtracing    [no test files]
?       github.com/open-policy-agent/opa/internal/file/archive  [no test files]
ok      github.com/open-policy-agent/opa/internal/file/url      0.003s
?       github.com/open-policy-agent/opa/internal/future        [no test files]
ok      github.com/open-policy-agent/opa/internal/gojsonschema  0.552s
?       github.com/open-policy-agent/opa/internal/ir    [no test files]
ok      github.com/open-policy-agent/opa/internal/ir/encoding   0.009s
ok      github.com/open-policy-agent/opa/internal/json/patch    0.005s
ok      github.com/open-policy-agent/opa/internal/jwx/buffer    0.002s
?       github.com/open-policy-agent/opa/internal/jwx/jwa       [no test files]
ok      github.com/open-policy-agent/opa/internal/jwx/jwk       0.006s
ok      github.com/open-policy-agent/opa/internal/jwx/jws       4.565s
ok      github.com/open-policy-agent/opa/internal/jwx/jws/sign  0.008s
ok      github.com/open-policy-agent/opa/internal/jwx/jws/verify        0.003s
ok      github.com/open-policy-agent/opa/internal/lcss  0.003s
ok      github.com/open-policy-agent/opa/internal/leb128        0.004s
ok      github.com/open-policy-agent/opa/internal/logging       0.004s
ok      github.com/open-policy-agent/opa/internal/merge 0.008s
ok      github.com/open-policy-agent/opa/internal/oracle        0.032s
ok      github.com/open-policy-agent/opa/internal/planner       0.022s
ok      github.com/open-policy-agent/opa/internal/presentation  0.012s
ok      github.com/open-policy-agent/opa/internal/prometheus    0.008s
?       github.com/open-policy-agent/opa/internal/ref   [no test files]
?       github.com/open-policy-agent/opa/internal/rego/opa      [no test files]
ok      github.com/open-policy-agent/opa/internal/report        0.013s
?       github.com/open-policy-agent/opa/internal/runtime       [no test files]
ok      github.com/open-policy-agent/opa/internal/runtime/init  0.042s
ok      github.com/open-policy-agent/opa/internal/semver        0.023s
?       github.com/open-policy-agent/opa/internal/storage/mock  [no test files]
?       github.com/open-policy-agent/opa/internal/strings       [no test files]
ok      github.com/open-policy-agent/opa/internal/strvals       0.014s
ok      github.com/open-policy-agent/opa/internal/uuid  0.018s
?       github.com/open-policy-agent/opa/internal/version       [no test files]
?       github.com/open-policy-agent/opa/internal/wasm/constant [no test files]
ok      github.com/open-policy-agent/opa/internal/wasm/encoding 0.040s
?       github.com/open-policy-agent/opa/internal/wasm/instruction      [no test files]
?       github.com/open-policy-agent/opa/internal/wasm/module   [no test files]
?       github.com/open-policy-agent/opa/internal/wasm/opcode   [no test files]
?       github.com/open-policy-agent/opa/internal/wasm/sdk/examples/basic       [no test files]
?       github.com/open-policy-agent/opa/internal/wasm/sdk/examples/loaders     [no test files]
ok      github.com/open-policy-agent/opa/internal/wasm/sdk/internal/wasm        8.345s
ok      github.com/open-policy-agent/opa/internal/wasm/sdk/opa  19.602s
?       github.com/open-policy-agent/opa/internal/wasm/sdk/opa/capabilities     [no test files]
?       github.com/open-policy-agent/opa/internal/wasm/sdk/opa/errors   [no test files]
?       github.com/open-policy-agent/opa/internal/wasm/sdk/opa/loader   [no test files]
ok      github.com/open-policy-agent/opa/internal/wasm/sdk/opa/loader/file      0.022s
ok      github.com/open-policy-agent/opa/internal/wasm/sdk/opa/loader/http      0.050s
?       github.com/open-policy-agent/opa/internal/wasm/types    [no test files]
?       github.com/open-policy-agent/opa/internal/wasm/util     [no test files]
ok      github.com/open-policy-agent/opa/keys   0.015s
ok      github.com/open-policy-agent/opa/loader 0.056s
ok      github.com/open-policy-agent/opa/logging        0.011s
?       github.com/open-policy-agent/opa/logging/test   [no test files]
ok      github.com/open-policy-agent/opa/metrics        0.015s
ok      github.com/open-policy-agent/opa/plugins        0.023s
ok      github.com/open-policy-agent/opa/plugins/bundle 4.295s
ok      github.com/open-policy-agent/opa/plugins/discovery      0.306s
ok      github.com/open-policy-agent/opa/plugins/logs   13.730s
ok      github.com/open-policy-agent/opa/plugins/rest   28.542s
ok      github.com/open-policy-agent/opa/plugins/status 1.084s
ok      github.com/open-policy-agent/opa/profiler       0.163s
ok      github.com/open-policy-agent/opa/refactor       0.022s
ok      github.com/open-policy-agent/opa/rego   14.689s
ok      github.com/open-policy-agent/opa/repl   2.280s
?       github.com/open-policy-agent/opa/resolver       [no test files]
?       github.com/open-policy-agent/opa/resolver/wasm  [no test files]
ok      github.com/open-policy-agent/opa/runtime        0.329s
ok      github.com/open-policy-agent/opa/sdk    0.343s
?       github.com/open-policy-agent/opa/sdk/test       [no test files]
ok      github.com/open-policy-agent/opa/server 4.507s
ok      github.com/open-policy-agent/opa/server/authorizer      0.201s
ok      github.com/open-policy-agent/opa/server/identifier      0.071s
?       github.com/open-policy-agent/opa/server/types   [no test files]
?       github.com/open-policy-agent/opa/server/writer  [no test files]
ok      github.com/open-policy-agent/opa/storage        0.018s
ok      github.com/open-policy-agent/opa/storage/disk   20.064s
ok      github.com/open-policy-agent/opa/storage/inmem  0.022s
?       github.com/open-policy-agent/opa/storage/internal/errors        [no test files]
?       github.com/open-policy-agent/opa/storage/internal/ptr   [no test files]
ok      github.com/open-policy-agent/opa/test/authz     0.310s
?       github.com/open-policy-agent/opa/test/cases     [no test files]
?       github.com/open-policy-agent/opa/test/e2e       [no test files]
ok      github.com/open-policy-agent/opa/test/e2e/authz 0.156s [no tests to run]
ok      github.com/open-policy-agent/opa/test/e2e/certrefresh   1.352s
ok      github.com/open-policy-agent/opa/test/e2e/concurrency   6.261s
ok      github.com/open-policy-agent/opa/test/e2e/diagnostics   0.280s
ok      github.com/open-policy-agent/opa/test/e2e/distributedtracing    0.240s
ok      github.com/open-policy-agent/opa/test/e2e/h2c   0.172s
?       github.com/open-policy-agent/opa/test/e2e/logs  [no test files]
ok      github.com/open-policy-agent/opa/test/e2e/logs/console  0.170s
ok      github.com/open-policy-agent/opa/test/e2e/metrics       0.216s
ok      github.com/open-policy-agent/opa/test/e2e/oci   3.164s
ok      github.com/open-policy-agent/opa/test/e2e/print 0.356s
ok      github.com/open-policy-agent/opa/test/e2e/shutdown      2.136s
ok      github.com/open-policy-agent/opa/test/e2e/tls   1.331s
ok      github.com/open-policy-agent/opa/test/e2e/wasm/authz    1.985s [no tests to run]
ok      github.com/open-policy-agent/opa/test/scheduler 0.145s
?       github.com/open-policy-agent/opa/test/wasm/cmd/wasm-rego-testgen        [no test files]
ok      github.com/open-policy-agent/opa/tester 33.035s
ok      github.com/open-policy-agent/opa/topdown        29.700s
?       github.com/open-policy-agent/opa/topdown/builtins       [no test files]
ok      github.com/open-policy-agent/opa/topdown/cache  0.023s
ok      github.com/open-policy-agent/opa/topdown/copypropagation        0.011s
ok      github.com/open-policy-agent/opa/topdown/lineage        0.051s
?       github.com/open-policy-agent/opa/topdown/print  [no test files]
?       github.com/open-policy-agent/opa/tracing        [no test files]
ok      github.com/open-policy-agent/opa/types  0.031s
ok      github.com/open-policy-agent/opa/util   0.011s
?       github.com/open-policy-agent/opa/util/test      [no test files]
?       github.com/open-policy-agent/opa/version        [no test files]
make[1]: Entering directory '/home/subham/opa/wasm'
make: '_obj/opa-test.wasm' is up to date.
make[1]: Leaving directory '/home/subham/opa/wasm'
GOVERSION=1.18 ./build/run-wasm-rego-tests.sh
+ GOVERSION=1.18
+ ASSETS=/home/subham/opa/test/wasm/assets
+ VERBOSE=0
+ TESTGEN_CONTAINER_NAME=opa-wasm-testgen-container
+ TESTRUN_CONTAINER_NAME=opa-wasm-testrun-container
+ main
+ trap interrupt SIGINT SIGTERM
+ mkdir -p /home/subham/opa/.go/cache/go-build
+ mkdir -p /home/subham/opa/.go/bin
+ generate_testcases
+ purge_testgen_container
+ docker kill opa-wasm-testgen-container
+ true
+ docker rm opa-wasm-testgen-container
++ id -u
++ id -g
+ docker run --name opa-wasm-testgen-container -u 0:0 -v /home/subham/opa/.go/bin:/go/bin -v /home/subham/opa:/src -v /home/subham/opa/test/wasm/assets:/assets -e GOCACHE=/src/.go/cache -w /src golang:1.18 sh -c 'make wasm-rego-testgen-install \
                && wasm-rego-testgen \
                --input-dir=/assets \
                --runner=/src/test/wasm/assets/test.js \
                --output=/src/.go/cache/testcases.tar.gz'
CGO_ENABLED=1 GOFLAGS="-buildmode=exe" go install ./test/wasm/cmd/wasm-rego-testgen
+ run_testcases
+ purge_testrun_container
+ docker kill opa-wasm-testrun-container
+ true
+ docker rm opa-wasm-testrun-container
+ true
+ wait 27130
+ docker run --rm --name opa-wasm-testrun-container --volumes-from opa-wasm-testgen-container -e VERBOSE=0 -w /scratch node:14 sh -c 'tar xzf \
            /src/.go/cache/testcases.tar.gz \
            && node test.js opa.wasm'
Found 369 WASM test cases in 19 file(s). Took 462.3995ms. Running now.

SUMMARY:
--------
PASS: 369/369

TOOK: 52.4289s
srenatus commented 2 years ago

Thanks a lot @odidev. This is certainly reassuring, but we can't expect you to manually run these things before each release 😅

Is there any way to run the tests on arm64 without maintaining manual GHA runners? Is there anything we could "plug in" and forget...? 🤔