search

krestomatio / keydb-operator

A KeyDB (Drop-In Alternative to Redis) Operator for Kubernetes, based on Ansible Operator SDK.
https://krestomatio.com
Apache License 2.0
53 stars 2 forks source link

Unable to retrieve the complete list of server APIs #145

Open oruchreis opened 6 months ago

oruchreis commented 6 months ago

Hi, I'm trying to test multimaster sample but I've got this error:

unable to retrieve the complete list of server APIs: packages.operators.coreos.com/v1: stale GroupVersion discovery: packages.operators.coreos.com/v1

Which example are you working with? keydb_v1alpha1_keydb_multimaster.yaml , operator version: 0.3.14

What is the current behavior? No Keydb instances created, there aren't any ss, pod etc. after applying sample.

Please tell us about your environment: Azure Aks (linux container) Kubernetes and nodepool version: 1.28.3 Storage class default: managed-premium

Log of keydb operator manager container

{"level":"info","ts":"2024-05-06T14:42:17Z","logger":"logging_event_handler","msg":"[playbook task start]","name":"keydb-sample-multimaster","namespace":"app-deps","gvk":"keydb.krestomat.io/v1alpha1, Kind=Keydb","event_type":"playbook_on_task_start","job":"5744481232734552105","EventData.Name":"krestomatio.k8s.keydb : add annotations"}
{"level":"error","ts":"2024-05-06T14:42:18Z","logger":"proxy","msg":"Unable to determine if virtual resource","gvk":"keydb.krestomat.io/v1alpha1, Kind=Keydb","error":"unable to retrieve the complete list of server APIs: packages.operators.coreos.com/v1: stale GroupVersion discovery: packages.operators.coreos.com/v1","stacktrace":"github.com/operator-framework/ansible-operator-plugins/internal/ansible/proxy.(*cacheResponseHandler).ServeHTTP\n\tansible-operator-plugins/internal/ansible/proxy/cache_response.go:99\nnet/http.serverHandler.ServeHTTP\n\t/opt/hostedtoolcache/go/1.20.11/x64/src/net/http/server.go:2936\nnet/http.(*conn).serve\n\t/opt/hostedtoolcache/go/1.20.11/x64/src/net/http/server.go:1995"}
--------------------------- Ansible Task StdOut -------------------------------
{"level":"info","ts":"2024-05-06T14:42:18Z","logger":"logging_event_handler","msg":"[playbook task start]","name":"keydb-sample-multimaster","namespace":"app-deps","gvk":"keydb.krestomat.io/v1alpha1, Kind=Keydb","event_type":"playbook_on_task_start","job":"5744481232734552105","EventData.Name":"krestomatio.k8s.keydb : failed msg"}
TASK [krestomatio.k8s.keydb : failed msg] **************************************
task path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/rescue.yml:4
-------------------------------------------------------------------------------
--------------------------- Ansible Task StdOut -------------------------------
 TASK [failed msg] ******************************** 
{"level":"error","ts":"2024-05-06T14:42:18Z","logger":"logging_event_handler","msg":"","name":"keydb-sample-multimaster","namespace":"app-deps","gvk":"keydb.krestomat.io/v1alpha1, Kind=Keydb","event_type":"runner_on_failed","job":"5744481232734552105","EventData.Task":"failed msg","EventData.TaskArgs":"","EventData.FailedTaskPath":"/opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/rescue.yml:4","error":"[playbook task failed]","stacktrace":"github.com/operator-framework/ansible-operator-plugins/internal/ansible/events.loggingEventHandler.Handle\n\tansible-operator-plugins/internal/ansible/events/log_events.go:111"}
fatal: [localhost]: FAILED! => {"changed": false, "msg": "There was an error before completing all tasks"}
-------------------------------------------------------------------------------
----- Ansible Task Status Event StdOut (keydb.krestomat.io/v1alpha1, Kind=Keydb, keydb-sample-multimaster/app-deps) -----
PLAY RECAP *********************************************************************
localhost                  : ok=10   changed=2    unreachable=0    failed=1    skipped=2    rescued=1    ignored=0   
----------
{"level":"error","ts":"2024-05-06T14:42:18Z","logger":"runner","msg":"ansible-playbook [core 2.15.6]\r\n  config file = /etc/ansible/ansible.cfg\r\n  configured module search path = ['/usr/share/ansible/openshift']\r\n  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible\r\n  ansible collection location = /opt/ansible/.ansible/collections:/usr/share/ansible/collections\r\n  executable location = /usr/local/bin/ansible-playbook\r\n  python version = 3.9.18 (main, Sep 22 2023, 17:58:34) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)] (/usr/bin/python3)\r\n  jinja version = 3.1.2\r\n  libyaml = True\r\nUsing /etc/ansible/ansible.cfg as config file\r\n[WARNING]: provided hosts list is empty, only localhost is available. Note that\r\nthe implicit localhost does not match 'all'\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/state/present/main.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/assertions.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/assertions.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/keydb.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/resource/secret.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/k8s/object.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/resource/cm.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/k8s/object.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/resource/svc-headless.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/k8s/object.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/resource/svc.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/k8s/object.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/resource/sts.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/resource/vpa.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/k8s/object.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/condition/ready/successful.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/routine/status.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/add-annotations.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/rescue.yml\r\nstatically imported: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/condition/ready/error.yml\r\nSkipping callback 'awx_display', as we already have a stdout callback.\nSkipping callback 'default', as we already have a stdout callback.\nSkipping callback 'minimal', as we already have a stdout callback.\nSkipping callback 'oneline', as we already have a stdout callback.\n\r\nPLAYBOOK: keydb.yml ************************************************************\n1 plays in /opt/ansible/playbooks/keydb.yml\n\r\nPLAY [keydb k8s resources] *****************************************************\n\r\nTASK [krestomatio.k8s.keydb : include keydb mode vars] *************************\r\ntask path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/main.yml:2\nok: [localhost] => {\"ansible_facts\": {\"keydb_args\": [\"--masterauth \\\"$(KEYDB_PASSWORD)\\\"\", \"--requirepass \\\"$(KEYDB_PASSWORD)\\\"\"], \"keydb_mode_config\": \"multi-master yes\\nactive-replica yes\\n\", \"keydb_size\": 3}, \"ansible_included_var_files\": [\"/opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/vars/multimaster.yml\"], \"changed\": false}\n\r\nTASK [krestomatio.k8s.keydb : assert k8s api-resources is not empty] ***********\r\ntask path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/assertions.yml:1\nok: [localhost] => {\"changed\": false, \"msg\": \"All assertions passed\"}\n\r\nTASK [krestomatio.k8s.keydb : assert k8s api-resources is not failed] **********\r\ntask path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/assertions.yml:9\nok: [localhost] => {\"changed\": false, \"msg\": \"All assertions passed\"}\n\r\nTASK [krestomatio.k8s.keydb : assert cr name max length] ***********************\r\ntask path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/assertions.yml:20\nok: [localhost] => {\"changed\": false, \"msg\": \"All assertions passed\"}\n\r\nTASK [krestomatio.k8s.keydb : keydb mode] **************************************\r\ntask path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/assertions.yml:4\nok: [localhost] => {\"changed\": false, \"msg\": \"All assertions passed\"}\n\r\nTASK [krestomatio.k8s.keydb : number of replicas if multimaster] ***************\r\ntask path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/assertions.yml:10\nok: [localhost] => {\"changed\": false, \"msg\": \"All assertions passed\"}\n\r\nTASK [krestomatio.k8s.keydb : resource definitions for keydb in state: present, kind: Secret] ***\r\ntask path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/k8s/object.yml:1\nredirecting (type: lookup) ansible.builtin.k8s to kubernetes.core.k8s\nfatal: [localhost]: FAILED! => {\"censored\": \"the output has been hidden due to the fact that 'no_log: true' was specified for this result\", \"changed\": false}\n\r\nTASK [krestomatio.k8s.keydb : set error status] ********************************\r\ntask path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/condition/ready/error.yml:1\nincluded: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/routine/status.yml for localhost\n\r\nTASK [krestomatio.k8s.keydb : set status] **************************************\r\ntask path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/routine/status.yml:1\nchanged: [localhost] => {\"changed\": true, \"result\": {\"apiVersion\": \"keydb.krestomat.io/v1alpha1\", \"kind\": \"Keydb\", \"metadata\": {\"creationTimestamp\": \"2024-05-06T14:42:14Z\", \"finalizers\": [\"keydb.krestomat.io/finalizer\"], \"generation\": 1, \"managedFields\": [{\"apiVersion\": \"keydb.krestomat.io/v1alpha1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:finalizers\": {\".\": {}, \"v:\\\"keydb.krestomat.io/finalizer\\\"\": {}}}}, \"manager\": \"ansible-operator\", \"operation\": \"Update\", \"time\": \"2024-05-06T14:42:14Z\"}, {\"apiVersion\": \"keydb.krestomat.io/v1alpha1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {}}, \"manager\": \"ansible-operator\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2024-05-06T14:42:14Z\"}, {\"apiVersion\": \"keydb.krestomat.io/v1alpha1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:spec\": {\".\": {}, \"f:keydbExtraConfig\": {}, \"f:keydbMode\": {}, \"f:keydbPvcDataAutoexpansion\": {}, \"f:keydbPvcDataAutoexpansionCapGib\": {}, \"f:keydbPvcDataAutoexpansionIncrementGib\": {}, \"f:keydbPvcDataSize\": {}, \"f:keydbResourceLimits\": {}, \"f:keydbResourceLimitsCpu\": {}, \"f:keydbResourceLimitsMemory\": {}}}, \"manager\": \"dashboard\", \"operation\": \"Update\", \"time\": \"2024-05-06T14:42:14Z\"}, {\"apiVersion\": \"keydb.krestomat.io/v1alpha1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\"f:conditions\": {}, \"f:observedGeneration\": {}, \"f:ready\": {}, \"f:state\": {}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2024-05-06T14:42:17Z\"}], \"name\": \"keydb-sample-multimaster\", \"namespace\": \"app-deps\", \"resourceVersion\": \"707446427\", \"uid\": \"6e1358c0-a0eb-4763-9678-9291be629084\"}, \"spec\": {\"keydbExtraConfig\": \"maxmemory 900mb\\nmaxmemory-policy allkeys-lru\\n\", \"keydbMode\": \"multimaster\", \"keydbPvcDataAutoexpansion\": true, \"keydbPvcDataAutoexpansionCapGib\": 25, \"keydbPvcDataAutoexpansionIncrementGib\": 5, \"keydbPvcDataSize\": \"1Gi\", \"keydbResourceLimits\": true, \"keydbResourceLimitsCpu\": 1, \"keydbResourceLimitsMemory\": \"1Gi\"}, \"status\": {\"conditions\": [{\"lastTransitionTime\": \"2024-05-06T14:42:14Z\", \"message\": \"Running reconciliation\", \"reason\": \"Running\", \"status\": \"True\", \"type\": \"Running\"}, {\"message\": \"There was an error before completing all tasks\", \"reason\": \"Error\", \"status\": \"False\", \"type\": \"Ready\"}], \"observedGeneration\": 1, \"ready\": \"False\", \"state\": \"Failed\"}}}\n\r\nTASK [krestomatio.k8s.keydb : notify status] ***********************************\r\ntask path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/routine/status.yml:27\nskipping: [localhost] => {\"changed\": false, \"false_condition\": \"_routine_status_cr_notify is defined and _routine_status_cr_notify\", \"skip_reason\": \"Conditional result was False\"}\n\r\nTASK [krestomatio.k8s.keydb : show notify status error] ************************\r\ntask path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/routine/status.yml:79\nskipping: [localhost] => {\"false_condition\": \"_routine_status_cr_notify is defined and _routine_status_cr_notify\"}\n\r\nTASK [krestomatio.k8s.keydb : add observed generation annotation] **************\r\ntask path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/condition/ready/error.yml:17\nincluded: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/add-annotations.yml for localhost\n\r\nTASK [krestomatio.k8s.keydb : add annotations] *********************************\r\ntask path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/add-annotations.yml:1\nchanged: [localhost] => {\"changed\": true, \"method\": \"update\", \"result\": {\"apiVersion\": \"keydb.krestomat.io/v1alpha1\", \"kind\": \"Keydb\", \"metadata\": {\"annotations\": {\"keydb.krestomat.io/observedGeneration\": \"1\", \"keydb.krestomat.io/ready\": \"False\"}, \"creationTimestamp\": \"2024-05-06T14:42:14Z\", \"finalizers\": [\"keydb.krestomat.io/finalizer\"], \"generation\": 1, \"managedFields\": [{\"apiVersion\": \"keydb.krestomat.io/v1alpha1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:finalizers\": {\".\": {}, \"v:\\\"keydb.krestomat.io/finalizer\\\"\": {}}}}, \"manager\": \"ansible-operator\", \"operation\": \"Update\", \"time\": \"2024-05-06T14:42:14Z\"}, {\"apiVersion\": \"keydb.krestomat.io/v1alpha1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {}}, \"manager\": \"ansible-operator\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2024-05-06T14:42:14Z\"}, {\"apiVersion\": \"keydb.krestomat.io/v1alpha1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:spec\": {\".\": {}, \"f:keydbExtraConfig\": {}, \"f:keydbMode\": {}, \"f:keydbPvcDataAutoexpansion\": {}, \"f:keydbPvcDataAutoexpansionCapGib\": {}, \"f:keydbPvcDataAutoexpansionIncrementGib\": {}, \"f:keydbPvcDataSize\": {}, \"f:keydbResourceLimits\": {}, \"f:keydbResourceLimitsCpu\": {}, \"f:keydbResourceLimitsMemory\": {}}}, \"manager\": \"dashboard\", \"operation\": \"Update\", \"time\": \"2024-05-06T14:42:14Z\"}, {\"apiVersion\": \"keydb.krestomat.io/v1alpha1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:status\": {\"f:conditions\": {}, \"f:observedGeneration\": {}, \"f:ready\": {}, \"f:state\": {}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"subresource\": \"status\", \"time\": \"2024-05-06T14:42:17Z\"}, {\"apiVersion\": \"keydb.krestomat.io/v1alpha1\", \"fieldsType\": \"FieldsV1\", \"fieldsV1\": {\"f:metadata\": {\"f:annotations\": {\".\": {}, \"f:keydb.krestomat.io/observedGeneration\": {}, \"f:keydb.krestomat.io/ready\": {}}}}, \"manager\": \"OpenAPI-Generator\", \"operation\": \"Update\", \"time\": \"2024-05-06T14:42:18Z\"}], \"name\": \"keydb-sample-multimaster\", \"namespace\": \"app-deps\", \"resourceVersion\": \"707446431\", \"uid\": \"6e1358c0-a0eb-4763-9678-9291be629084\"}, \"spec\": {\"keydbExtraConfig\": \"maxmemory 900mb\\nmaxmemory-policy allkeys-lru\\n\", \"keydbMode\": \"multimaster\", \"keydbPvcDataAutoexpansion\": true, \"keydbPvcDataAutoexpansionCapGib\": 25, \"keydbPvcDataAutoexpansionIncrementGib\": 5, \"keydbPvcDataSize\": \"1Gi\", \"keydbResourceLimits\": true, \"keydbResourceLimitsCpu\": 1, \"keydbResourceLimitsMemory\": \"1Gi\"}, \"status\": {\"conditions\": [{\"lastTransitionTime\": \"2024-05-06T14:42:14Z\", \"message\": \"Running reconciliation\", \"reason\": \"Running\", \"status\": \"True\", \"type\": \"Running\"}, {\"message\": \"There was an error before completing all tasks\", \"reason\": \"Error\", \"status\": \"False\", \"type\": \"Ready\"}], \"observedGeneration\": 1, \"ready\": \"False\", \"state\": \"Failed\"}}}\n\r\nTASK [krestomatio.k8s.keydb : failed msg] **************************************\r\ntask path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/rescue.yml:4\nfatal: [localhost]: FAILED! => {\"changed\": false, \"msg\": \"There was an error before completing all tasks\"}\n\r\nPLAY RECAP *********************************************************************\r\nlocalhost                  : ok=10   changed=2    unreachable=0    failed=1    skipped=2    rescued=1    ignored=0   \n","job":"5744481232734552105","name":"keydb-sample-multimaster","namespace":"app-deps","error":"exit status 2","stacktrace":"github.com/operator-framework/ansible-operator-plugins/internal/ansible/runner.(*runner).Run.func1\n\tansible-operator-plugins/internal/ansible/runner/runner.go:269"}
{"level":"error","ts":"2024-05-06T14:42:18Z","msg":"Reconciler error","controller":"keydb-controller","object":{"name":"keydb-sample-multimaster","namespace":"app-deps"},"namespace":"app-deps","name":"keydb-sample-multimaster","reconcileID":"dec81d85-852d-45b9-b693-5d3203d40160","error":"event runner on failed","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.3/pkg/internal/controller/controller.go:324\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.3/pkg/internal/controller/controller.go:265\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.3/pkg/internal/controller/controller.go:226"}
{"level":"error","ts":"2024-05-06T14:42:19Z","logger":"proxy","msg":"Unable to determine if virtual resource","gvk":"/v1, Kind=Pod","error":"unable to retrieve the complete list of server APIs: packages.operators.coreos.com/v1: stale GroupVersion discovery: packages.operators.coreos.com/v1","stacktrace":"github.com/operator-framework/ansible-operator-plugins/internal/ansible/proxy.(*cacheResponseHandler).ServeHTTP\n\tansible-operator-plugins/internal/ansible/proxy/cache_response.go:99\nnet/http.serverHandler.ServeHTTP\n\t/opt/hostedtoolcache/go/1.20.11/x64/src/net/http/server.go:2936\nnet/http.(*conn).serve\n\t/opt/hostedtoolcache/go/1.20.11/x64/src/net/http/server.go:1995"}
{"level":"error","ts":"2024-05-06T14:42:19Z","logger":"proxy","msg":"Unable to determine if virtual resource","gvk":"keydb.krestomat.io/v1alpha1, Kind=Keydb","error":"unable to retrieve the complete list of server APIs: packages.operators.coreos.com/v1: stale GroupVersion discovery: packages.operators.coreos.com/v1","stacktrace":"github.com/operator-framework/ansible-operator-plugins/internal/ansible/proxy.(*cacheResponseHandler).ServeHTTP\n\tansible-operator-plugins/internal/ansible/proxy/cache_response.go:99\nnet/http.serverHandler.ServeHTTP\n\t/opt/hostedtoolcache/go/1.20.11/x64/src/net/http/server.go:2936\nnet/http.(*conn).serve\n\t/opt/hostedtoolcache/go/1.20.11/x64/src/net/http/server.go:1995"}
--------------------------- Ansible Task StdOut -------------------------------
TASK [krestomatio.k8s.keydb : include keydb mode vars] *************************
task path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/main.yml:2
-------------------------------------------------------------------------------
{"level":"info","ts":"2024-05-06T14:42:19Z","logger":"logging_event_handler","msg":"[playbook task start]","name":"keydb-sample-multimaster","namespace":"app-deps","gvk":"keydb.krestomat.io/v1alpha1, Kind=Keydb","event_type":"playbook_on_task_start","job":"1639556587148026168","EventData.Name":"krestomatio.k8s.keydb : include keydb mode vars"}
--------------------------- Ansible Task StdOut -------------------------------
{"level":"info","ts":"2024-05-06T14:42:19Z","logger":"logging_event_handler","msg":"[playbook task start]","name":"keydb-sample-multimaster","namespace":"app-deps","gvk":"keydb.krestomat.io/v1alpha1, Kind=Keydb","event_type":"playbook_on_task_start","job":"1639556587148026168","EventData.Name":"krestomatio.k8s.keydb : assert k8s api-resources is not empty"}
TASK [krestomatio.k8s.keydb : assert k8s api-resources is not empty] ***********
task path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/assertions.yml:1
-------------------------------------------------------------------------------
--------------------------- Ansible Task StdOut -------------------------------
TASK [krestomatio.k8s.keydb : assert k8s api-resources is not failed] **********
task path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/assertions.yml:9
-------------------------------------------------------------------------------
{"level":"info","ts":"2024-05-06T14:42:19Z","logger":"logging_event_handler","msg":"[playbook task start]","name":"keydb-sample-multimaster","namespace":"app-deps","gvk":"keydb.krestomat.io/v1alpha1, Kind=Keydb","event_type":"playbook_on_task_start","job":"1639556587148026168","EventData.Name":"krestomatio.k8s.keydb : assert k8s api-resources is not failed"}
{"level":"info","ts":"2024-05-06T14:42:19Z","logger":"logging_event_handler","msg":"[playbook task start]","name":"keydb-sample-multimaster","namespace":"app-deps","gvk":"keydb.krestomat.io/v1alpha1, Kind=Keydb","event_type":"playbook_on_task_start","job":"1639556587148026168","EventData.Name":"krestomatio.k8s.keydb : assert cr name max length"}
--------------------------- Ansible Task StdOut -------------------------------
TASK [krestomatio.k8s.keydb : assert cr name max length] ***********************
task path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/assertions.yml:20
-------------------------------------------------------------------------------
--------------------------- Ansible Task StdOut -------------------------------
TASK [krestomatio.k8s.keydb : keydb mode] **************************************
task path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/assertions.yml:4
{"level":"info","ts":"2024-05-06T14:42:19Z","logger":"logging_event_handler","msg":"[playbook task start]","name":"keydb-sample-multimaster","namespace":"app-deps","gvk":"keydb.krestomat.io/v1alpha1, Kind=Keydb","event_type":"playbook_on_task_start","job":"1639556587148026168","EventData.Name":"krestomatio.k8s.keydb : keydb mode"}
-------------------------------------------------------------------------------
--------------------------- Ansible Task StdOut -------------------------------
{"level":"info","ts":"2024-05-06T14:42:20Z","logger":"logging_event_handler","msg":"[playbook task start]","name":"keydb-sample-multimaster","namespace":"app-deps","gvk":"keydb.krestomat.io/v1alpha1, Kind=Keydb","event_type":"playbook_on_task_start","job":"1639556587148026168","EventData.Name":"krestomatio.k8s.keydb : number of replicas if multimaster"}
TASK [krestomatio.k8s.keydb : number of replicas if multimaster] ***************
jobcespedes commented 6 months ago

Hi,

Would you please share the method/steps used to install the keydb operator in the cluster?

oruchreis commented 6 months ago

Hi, I have installed as described in the document. Also I've reinstalled operator 3 times, but no luck. Exact steps are:

aliseckinb commented 6 months ago

I am encountering the same issue.

jobcespedes commented 6 months ago

@oruchreis thanks for the info. Does the error also occur in standalone mode?

From the installation method and shared logs, there is a message that could lead us to the cause. Secret creation is failing:

TASK [krestomatio.k8s.keydb : resource definitions for keydb in state: present, kind: Secret] ***
task path: /opt/ansible/.ansible/collections/ansible_collections/krestomatio/k8s/roles/v1alpha1/database/keydb/tasks/common/k8s/object.yml:1
redirecting (type: lookup) ansible.builtin.k8s to kubernetes.core.k8s
fatal: [localhost]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}

The details of the Secrets steps are hidden to prevent sensitive information from leaking into the logs. I would need to recreate a dev environment as similar as possible, to dig deeper.

oruchreis commented 6 months ago

@jobcespedes yes, I've tried standalone mode as described in the documentation mode and the error occurs in standalone mode, too. The Crds created but there are no objects in the crds.

> kubectl get keydb
NAME                       AGE   STATUS   SINCE   MODE          SERVICE
keydb-sample               20m   Failed   20m     standalone
keydb-sample-multimaster   20d   Failed   20d     multimaster
> kubectl describe keydb keydb-sample
Name:         keydb-sample
Namespace:    **
Labels:       <none>
Annotations:  keydb.krestomat.io/observedGeneration: 1
              keydb.krestomat.io/ready: False
API Version:  keydb.krestomat.io/v1alpha1
Kind:         Keydb
Metadata:
  Creation Timestamp:  2024-05-27T06:34:41Z
  Finalizers:
    keydb.krestomat.io/finalizer
  Generation:  1
  Managed Fields:
    API Version:  keydb.krestomat.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:finalizers:
          .:
          v:"keydb.krestomat.io/finalizer":
    Manager:      ansible-operator
    Operation:    Update
    Time:         2024-05-27T06:34:41Z
    API Version:  keydb.krestomat.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:spec:
        .:
        f:keydbExtraConfig:
        f:keydbMode:
        f:keydbPvcDataSize:
        f:keydbResourceLimits:
        f:keydbResourceLimitsCpu:
        f:keydbResourceLimitsMemory:
    Manager:      dashboard
    Operation:    Update
    Time:         2024-05-27T06:34:41Z
    API Version:  keydb.krestomat.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        f:observedGeneration:
        f:ready:
        f:state:
    Manager:      OpenAPI-Generator
    Operation:    Update
    Subresource:  status
    Time:         2024-05-27T06:34:44Z
    API Version:  keydb.krestomat.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:keydb.krestomat.io/observedGeneration:
          f:keydb.krestomat.io/ready:
    Manager:      OpenAPI-Generator
    Operation:    Update
    Time:         2024-05-27T06:34:45Z
    API Version:  keydb.krestomat.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:conditions:
    Manager:         ansible-operator
    Operation:       Update
    Subresource:     status
    Time:            2024-05-27T06:46:52Z
  Resource Version:  733901325
  UID:               341fa506-e309-49ec-a97e-6bfe161eed26
Spec:
  Keydb Extra Config:  maxmemory 900mb
maxmemory-policy allkeys-lru

  Keydb Mode:                    standalone
  Keydb Pvc Data Size:           1Gi
  Keydb Resource Limits:         true
  Keydb Resource Limits Cpu:     1
  Keydb Resource Limits Memory:  1Gi
Status:
  Conditions:
    Last Transition Time:  2024-05-27T06:34:45Z
    Message:               There was an error before completing all tasks
    Reason:                Error
    Status:                False
    Type:                  Ready
    Ansible Result:
      Changed:             2
      Completion:          2024-05-27T06:34:45.323368
      Failures:            1
      Ok:                  9
      Skipped:             3
    Last Transition Time:  2024-05-27T06:34:45Z
    Message:               unknown playbook failure
There was an error before completing all tasks
    Reason:                Failed
    Status:                True
    Type:                  Failure
    Last Transition Time:  2024-05-27T06:34:45Z
    Message:
    Reason:
    Status:                False
    Type:                  Successful
    Last Transition Time:  2024-05-27T06:46:48Z
    Message:               Running reconciliation
    Reason:                Running
    Status:                False
    Type:                  Running
  Observed Generation:     1
  Ready:                   False
  State:                   Failed
Events:                    <none>
aliseckinb commented 5 months ago

@jobcespedes hello Do you have any progress?

jobcespedes commented 5 months ago

Unfortunately not yet. I haven't found time to create an environment with the same conditions, in Azure Provider.