mwangggg commented 3 months ago

Welcome to Cryostat! 👋

Before contributing, make sure you have:

[x] Read the contributing guidelines
[x] Linked a relevant issue which this PR resolves
[x] Linked any other relevant issues, PR's, or documentation, if any
[x] Resolved all conflicts, if any
[x] Rebased your branch PR on top of the latest upstream main branch
[x] Attached at least one of the following labels to the PR: [chore, ci, docs, feat, fix, test]
[x] Signed all commits: git commit -S -m "YOUR_COMMIT_MESSAGE"

Fixes: #814 Depends on #968

andrewazores commented 2 weeks ago

/build_test

andrewazores commented 2 weeks ago

https://github.com/cryostatio/cryostat-operator/actions/runs/11631338954

github-actions[bot] commented 2 weeks ago

/build_test : At least one test failed ❌. View Actions Run.

andrewazores commented 2 weeks ago

@ebaron every scorecard test in the run seems to have state: pass, but the log ends like this:

        The heap is around 0.227 % full. There is likely no big benefit from enabling string deduplication.] name:String Deduplication score:12.901007290315437 topic:heap] SystemGc:map[evaluation:map[explanation:<nil> solution:<nil> suggestions:[] summary:No garbage collections were caused by System.gc().] name:GCs Caused by System.gc() score:0 topic:garbage_collection] TlabAllocationRatio:map[evaluation:map[explanation:<nil> solution:Allocating objects outside of Thread Local Allocation Buffers (TLABs) is more expensive than allocating inside TLABs. This may be acceptable if the individual allocations are intended to be larger than a reasonable TLAB. It may be possible to avoid this by decreasing the size of the individual allocations. There are some TLAB related JVM flags that you can experiment with, but it is usually better to let the JVM manage TLAB sizes automatically. suggestions:[] summary:The program allocated 48.4 % of the memory outside of TLABs.] name:TLAB Allocation Ratio score:46.712468530433846 topic
namespace "cryostat-operator-scorecard" deleted
make: *** [Makefile:199: test-scorecard] Error 1
Error: Process completed with exit code 2.

(that first piece is an automated analysis rule result snippet)

any idea what that's about? Is it a timeout?

andrewazores commented 2 weeks ago

/build_test

github-actions[bot] commented 2 weeks ago

/build_test : At least one test failed ❌. View Actions Run.

ebaron commented 2 weeks ago

Looks like this one failed due to timeout in the most recent run:

2024-11-04T15:11:20.0968904Z Image:      ghcr.io/cryostatio/cryostat-operator-scorecard:pr-923-0ec9a84ffded9865671402790379c2884097f456
2024-11-04T15:11:20.0969353Z Entrypoint: [cryostat-scorecard-tests cryostat-multi-namespace]
2024-11-04T15:11:20.0969479Z Labels:
2024-11-04T15:11:20.0969613Z    "suite":"cryostat"
2024-11-04T15:11:20.0969841Z    "test":"cryostat-multi-namespace"
2024-11-04T15:11:20.0969969Z Results:
2024-11-04T15:11:20.0970194Z    Name: cryostat-multi-namespace
2024-11-04T15:11:20.0970323Z    State: fail
2024-11-04T15:11:20.0970330Z 
2024-11-04T15:11:20.0970456Z    Errors:
2024-11-04T15:11:20.0970899Z        Cryostat main deployment did not become available: context deadline exceeded
2024-11-04T15:11:20.0971548Z        cryostat-multi-namespace test failed: context deadline exceeded

andrewazores commented 2 weeks ago

Weird. I can reproduce the failure locally with make test-scorecard, but make test-scorecard-local SCORECARD_TEST_SELECTION=operator-install,cryostat-cr,cryostat-multi-namespace,cryostat-recording,cryostat-config-change,cryostat-report seems to succeed.

andrewazores commented 2 weeks ago

NAME                                                                  READY   STATUS      RESTARTS   AGE
pod/andrewazores-cryostat-operator-bundle-4-0-0-split-deployments-4   1/1     Running     0          7m1s
pod/cryostat-operator-controller-5559bcd8b6-vm6k4                     1/1     Running     0          6m42s
pod/fa200ffb618b00c1795a2635f60894bdc36cfbb76a6db4f8c2748a6056gqnvm   0/1     Completed   0          6m57s
pod/scorecard-test-5n99                                               0/1     Completed   0          6m29s
pod/scorecard-test-7kxq                                               0/1     Completed   0          6m29s
pod/scorecard-test-f7lw                                               0/1     Completed   0          6m29s
pod/scorecard-test-hmxx                                               1/1     Running     0          5m48s
pod/scorecard-test-jq6r                                               0/1     Completed   0          6m22s
pod/scorecard-test-kjb2                                               0/1     Completed   0          6m29s
pod/scorecard-test-r7jb                                               0/1     Completed   0          6m25s
pod/scorecard-test-vtkt                                               0/1     Completed   0          6m29s
pod/scorecard-test-xbmc                                               0/1     Completed   0          6m29s

NAME                                           TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
service/cryostat-operator-controller-service   ClusterIP   10.217.5.60   <none>        443/TCP   6m42s
service/cryostat-operator-webhook-service      ClusterIP   10.217.5.23   <none>        443/TCP   6m45s

NAME                                           READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/cryostat-operator-controller   1/1     1            1           6m42s

NAME                                                      DESIRED   CURRENT   READY   AGE
replicaset.apps/cryostat-operator-controller-5559bcd8b6   1         1         1       6m42s

NAME                                                                        STATUS     COMPLETIONS   DURATION   AGE
job.batch/fa200ffb618b00c1795a2635f60894bdc36cfbb76a6db4f8c2748a60563e689   Complete   1/1           8s         6m57s

function cleanup { ( set +e; /home/work/bin/operator-sdk cleanup -n cryostat-operator-scorecard cryostat-operator; /home/work/workspace/cryostat-operator/bin/kustomize build internal/images/custom-scorecard-tests/rbac/ | oc delete --ignore-not-found=false -f -; oc delete --ignore-not-found=false -n cryostat-operator-scorecard secret registry-key; oc delete --ignore-not-found=false namespace cryostat-operator-scorecard; ) }; cleanup
WARN[0000] Cleanup operator: package "cryostat-operator" not found 
Error from server (NotFound): error when deleting "STDIN": serviceaccounts "cryostat-scorecard" not found
Error from server (NotFound): error when deleting "STDIN": roles.rbac.authorization.k8s.io "cryostat-scorecard" not found
Error from server (NotFound): error when deleting "STDIN": clusterroles.rbac.authorization.k8s.io "cryostat-scorecard" not found
Error from server (NotFound): error when deleting "STDIN": rolebindings.rbac.authorization.k8s.io "cryostat-scorecard" not found
Error from server (NotFound): error when deleting "STDIN": clusterrolebindings.rbac.authorization.k8s.io "cryostat-scorecard" not found
Error from server (NotFound): secrets "registry-key" not found
Error from server (NotFound): namespaces "cryostat-operator-scorecard" not found
make: [Makefile:221: clean-scorecard] Error 1 (ignored)
mkdir -p /home/work/workspace/cryostat-operator/bin
test -s /home/work/workspace/cryostat-operator/bin/kustomize || { curl -Ss "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash -s -- 4.5.7 /home/work/workspace/cryostat-operator/bin; }
oc create namespace cryostat-operator-scorecard && oc label --overwrite namespace cryostat-operator-scorecard pod-security.kubernetes.io/warn=restricted pod-security.kubernetes.io/audit=restricted
namespace/cryostat-operator-scorecard created
namespace/cryostat-operator-scorecard labeled
cd internal/images/custom-scorecard-tests/rbac/ && /home/work/workspace/cryostat-operator/bin/kustomize edit set namespace cryostat-operator-scorecard
/home/work/workspace/cryostat-operator/bin/kustomize build internal/images/custom-scorecard-tests/rbac/ | oc apply -f -
serviceaccount/cryostat-scorecard created
role.rbac.authorization.k8s.io/cryostat-scorecard created
clusterrole.rbac.authorization.k8s.io/cryostat-scorecard created
rolebinding.rbac.authorization.k8s.io/cryostat-scorecard created
clusterrolebinding.rbac.authorization.k8s.io/cryostat-scorecard created
/home/work/bin/operator-sdk run bundle -n cryostat-operator-scorecard --timeout 20m quay.io/andrewazores/cryostat-operator-bundle:4.0.0-split-deployments-4 --security-context-config=restricted 
INFO[0016] Creating a File-Based Catalog of the bundle "quay.io/andrewazores/cryostat-operator-bundle:4.0.0-split-deployments-4" 
INFO[0016] Generated a valid File-Based Catalog         
INFO[0019] Created registry pod: andrewazores-cryostat-operator-bundle-4-0-0-split-deployments-4 
INFO[0019] Created CatalogSource: cryostat-operator-catalog 
INFO[0019] OperatorGroup "operator-sdk-og" created      
INFO[0019] Created Subscription: cryostat-operator-v4-0-0-split-deployments-4-sub 
INFO[0030] Approved InstallPlan install-qzqrx for the Subscription: cryostat-operator-v4-0-0-split-deployments-4-sub 
INFO[0030] Waiting for ClusterServiceVersion "cryostat-operator-scorecard/cryostat-operator.v4.0.0-split-deployments-4" to reach 'Succeeded' phase 
INFO[0031]   Waiting for ClusterServiceVersion "cryostat-operator-scorecard/cryostat-operator.v4.0.0-split-deployments-4" to appear 
INFO[0033]   Found ClusterServiceVersion "cryostat-operator-scorecard/cryostat-operator.v4.0.0-split-deployments-4" phase: Pending 
INFO[0036]   Found ClusterServiceVersion "cryostat-operator-scorecard/cryostat-operator.v4.0.0-split-deployments-4" phase: Installing 
INFO[0037]   Found ClusterServiceVersion "cryostat-operator-scorecard/cryostat-operator.v4.0.0-split-deployments-4" phase: InstallReady 
INFO[0038]   Found ClusterServiceVersion "cryostat-operator-scorecard/cryostat-operator.v4.0.0-split-deployments-4" phase: Installing 
INFO[0047]   Found ClusterServiceVersion "cryostat-operator-scorecard/cryostat-operator.v4.0.0-split-deployments-4" phase: Succeeded 
INFO[0047] OLM has successfully installed "cryostat-operator.v4.0.0-split-deployments-4" 
function cleanup { ( set +e; /home/work/bin/operator-sdk cleanup -n cryostat-operator-scorecard cryostat-operator; /home/work/workspace/cryostat-operator/bin/kustomize build internal/images/custom-scorecard-tests/rbac/ | oc delete --ignore-not-found=false -f -; oc delete --ignore-not-found=false -n cryostat-operator-scorecard secret registry-key; oc delete --ignore-not-found=false namespace cryostat-operator-scorecard; ) } ; \
trap cleanup EXIT ; \
/home/work/bin/operator-sdk scorecard -n cryostat-operator-scorecard -s cryostat-scorecard -w 20m quay.io/andrewazores/cryostat-operator-bundle:4.0.0-split-deployments-4 --pod-security=restricted

$ oc logs -n cryostat-operator-scorecard -f pod/cryostat-operator-controller-5559bcd8b6-vm6k4

2024-11-04T21:17:55Z    ERROR   controllers.Cryostat    Failed to set up TLS for Cryostat   {"Request.Namespace": "cryostat-operator-scorecard", "Request.Name": "cryostat-multi-namespace", "error": "admission webhook \"webhook.cert-manager.io\" denied the request: spec.commonName: Too long: must have at most 64 bytes"}
github.com/cryostatio/cryostat-operator/internal/controllers.(*Reconciler).reconcileCryostat
    /workspace/internal/controllers/reconciler.go:249
github.com/cryostatio/cryostat-operator/internal/controllers.(*CryostatReconciler).Reconcile
    /workspace/internal/controllers/cryostat_controller.go:102
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.6/pkg/internal/controller/controller.go:119
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.6/pkg/internal/controller/controller.go:316
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.6/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.6/pkg/internal/controller/controller.go:227
2024-11-04T21:17:55Z    ERROR   Reconciler error    {"controller": "cryostat", "controllerGroup": "operator.cryostat.io", "controllerKind": "Cryostat", "Cryostat": {"name":"cryostat-multi-namespace","namespace":"cryostat-operator-scorecard"}, "namespace": "cryostat-operator-scorecard", "name": "cryostat-multi-namespace", "reconcileID": "4d033dc6-013b-408e-8fdf-92725ab727ae", "error": "admission webhook \"webhook.cert-manager.io\" denied the request: spec.commonName: Too long: must have at most 64 bytes"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.6/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.6/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.6/pkg/internal/controller/controller.go:227
2024-11-04T21:17:59Z    INFO    controllers.Cryostat    Reconciling Cryostat    {"Request.Namespace": "cryostat-operator-scorecard", "Request.Name": "cryostat-cr"}
2024-11-04T21:17:59Z    INFO    controllers.Cryostat    Cryostat instance not found {"Request.Namespace": "cryostat-operator-scorecard", "Request.Name": "cryostat-cr"}

andrewazores commented 2 weeks ago

964 ^

github-actions[bot] commented 2 weeks ago

This PR/issue depends on:

cryostatio/cryostat-operator#968 By Dependent Issues (🤖). Happy coding!

andrewazores commented 1 week ago

Last two commits include #968 and an accompanying adjustment.

andrewazores commented 1 week ago

/build_test

github-actions[bot] commented 1 week ago

/build_test completed successfully ✅. View Actions Run.

cryostatio / cryostat-operator

feat(deploy): separate -storage and -db pods #923

Welcome to Cryostat! 👋

Before contributing, make sure you have:

964 ^