search

noobaa / noobaa-operator

Operator for NooBaa - object data service for hybrid and multi cloud environments :cloud: :wrench:
https://www.noobaa.io
Apache License 2.0
104 stars 101 forks source link

NooBaa instance Stuck on Configuring phase #682

Open m-usmanayub opened 3 years ago

m-usmanayub commented 3 years ago

Hi Folks,

I am trying to setup a simple MCG for Red Hat Quay registry using the following guide:

https://access.redhat.com/documentation/en-us/red_hat_quay/3.5/html/deploy_red_hat_quay_on_openshift_with_the_quay_operator/installing_the_quay_operator

After the creation of noobaa instance and while trying to find the status using the command, I see that it is stuck on Configuring phase

oc get -n openshift-storage noobaas noobaa -w
NAME     MGMT-ENDPOINTS   S3-ENDPOINTS   IMAGE                                                                                                            PHASE      AGE
noobaa                                   registry.redhat.io/ocs4/mcg-core-rhel8@sha256:6ff8645efdde95fa97d496084d3555b7680895f0b79c147f2a880b43742af3a4   Creating   5s
noobaa                                   registry.redhat.io/ocs4/mcg-core-rhel8@sha256:6ff8645efdde95fa97d496084d3555b7680895f0b79c147f2a880b43742af3a4   Connecting   7s
noobaa   ["https://10.150.8.166:31268"]                  registry.redhat.io/ocs4/mcg-core-rhel8@sha256:6ff8645efdde95fa97d496084d3555b7680895f0b79c147f2a880b43742af3a4   Connecting   11s
noobaa   ["https://10.150.8.166:31268"]                  registry.redhat.io/ocs4/mcg-core-rhel8@sha256:6ff8645efdde95fa97d496084d3555b7680895f0b79c147f2a880b43742af3a4   Connecting   14s
noobaa   ["https://10.150.8.166:31268"]                  registry.redhat.io/ocs4/mcg-core-rhel8@sha256:6ff8645efdde95fa97d496084d3555b7680895f0b79c147f2a880b43742af3a4   Configuring   22s

Upon checking the logs on the noobaa-core pod I find that there is an Error Credentials not found

oc logs noobaa-core-0 | tail -100
Jul-8 5:18:38.624 [WebServer/36]    [L0] core.server.system_services.redirector:: publish_to_cluster: server_inter_process update_master_change { is_master: true } [ 'fcall://fcall(2lq5x8u)', 'ws://[::ffff:127.0.0.1]:44874/(4dep0fq)', 'ws://[::ffff:127.0.0.1]:44962/(4ouj6fn)' ]
Jul-8 5:18:39.395 [WebServer/36]    [L0] core.server.common_services.auth_server:: credentials account not found SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:18:39.424 [BGWorkers/32]    [L0] core.server.bg_services.agent_blocks_reclaimer:: AGENT_BLOCKS_RECLAIMER: BEGIN
Jul-8 5:18:39.503 [WebServer/36]    [L0] core.server.common_services.auth_server:: password mismatch SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:18:39.504 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv auth_api.create_auth reqid wss://noobaa-mgmt.openshift-storage.svc.cluster.local:443/rpc/-1646 connid ws://[::ffff:10.130.0.148]:39970/(2lkyyis) Error: credentials not found
    at /root/node_modules/noobaa-core/src/server/common_services/auth_server.js:71:39
    at async /root/node_modules/noobaa-core/src/rpc/rpc.js:340:32
    at async RPC._on_request (/root/node_modules/noobaa-core/src/rpc/rpc.js:348:25)
Jul-8 5:18:42.919 [WebServer/36]    [L0] core.server.common_services.auth_server:: credentials account not found SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:18:43.019 [WebServer/36]    [L0] core.server.common_services.auth_server:: password mismatch SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:18:43.019 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv auth_api.create_auth reqid wss://noobaa-mgmt.openshift-storage.svc.cluster.local:443/rpc/-1649 connid ws://[::ffff:10.130.0.148]:39970/(2lkyyis) Error: credentials not found
    at /root/node_modules/noobaa-core/src/server/common_services/auth_server.js:71:39
    at async /root/node_modules/noobaa-core/src/rpc/rpc.js:340:32
    at async RPC._on_request (/root/node_modules/noobaa-core/src/rpc/rpc.js:348:25)
Jul-8 5:18:46.200 [WebServer/36]    [L0] core.server.common_services.auth_server:: credentials account not found SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:18:46.299 [WebServer/36]    [L0] core.server.common_services.auth_server:: password mismatch SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:18:46.300 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv auth_api.create_auth reqid wss://noobaa-mgmt.openshift-storage.svc.cluster.local:443/rpc/-1652 connid ws://[::ffff:10.130.0.148]:39970/(2lkyyis) Error: credentials not found
    at /root/node_modules/noobaa-core/src/server/common_services/auth_server.js:71:39
    at async /root/node_modules/noobaa-core/src/rpc/rpc.js:340:32
    at async RPC._on_request (/root/node_modules/noobaa-core/src/rpc/rpc.js:348:25)
Jul-8 5:18:47.324 [BGWorkers/32]    [L0] core.server.bg_services.mirror_writer:: no buckets with mirror policy. nothing to do
Jul-8 5:18:47.325 [BGWorkers/32]    [L0] core.server.bg_services.buckets_reclaimer:: no buckets in "deleting" state. nothing to do
Jul-8 5:18:47.326 [BGWorkers/32]   [LOG] CONSOLE:: TieringTTFWorker: start running
Jul-8 5:18:47.326 [BGWorkers/32]    [L0] core.server.bg_services.tier_ttf_worker:: no buckets with more than one tier. nothing to do
Jul-8 5:18:47.381 [BGWorkers/32]   [LOG] CONSOLE:: TieringSpillbackWorker: start running
Jul-8 5:18:48.628 [BGWorkers/32]    [L0] core.server.bg_services.cluster_master:: no local cluster info or server is not part of a cluster. therefore will be cluster master
Jul-8 5:18:48.632 [WebServer/36]    [L0] core.server.system_services.redirector:: publish_to_cluster: server_inter_process update_master_change { is_master: true } [ 'fcall://fcall(2lq5x8u)', 'ws://[::ffff:127.0.0.1]:44874/(4dep0fq)', 'ws://[::ffff:127.0.0.1]:44962/(4ouj6fn)' ]
Jul-8 5:18:49.132 [BGWorkers/32]    [L0] core.server.bg_services.scrubber:: SCRUBBER: BEGIN
Jul-8 5:18:49.136 [BGWorkers/32]    [L0] core.server.bg_services.scrubber:: SCRUBBER: END
Jul-8 5:18:49.439 [BGWorkers/32]    [L0] core.server.bg_services.agent_blocks_reclaimer:: AGENT_BLOCKS_RECLAIMER: BEGIN
Jul-8 5:18:49.453 [WebServer/36]    [L0] core.server.common_services.auth_server:: credentials account not found SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:18:49.552 [WebServer/36]    [L0] core.server.common_services.auth_server:: password mismatch SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:18:49.553 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv auth_api.create_auth reqid wss://noobaa-mgmt.openshift-storage.svc.cluster.local:443/rpc/-1655 connid ws://[::ffff:10.130.0.148]:39970/(2lkyyis) Error: credentials not found
    at /root/node_modules/noobaa-core/src/server/common_services/auth_server.js:71:39
    at async /root/node_modules/noobaa-core/src/rpc/rpc.js:340:32
    at async RPC._on_request (/root/node_modules/noobaa-core/src/rpc/rpc.js:348:25)
Jul-8 5:18:50.502 [BGWorkers/32]    [L0] core.server.bg_services.objects_reclaimer:: no objects in "unreclaimed" state. nothing to do
Jul-8 5:18:52.721 [WebServer/36]    [L0] core.server.common_services.auth_server:: credentials account not found SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:18:52.819 [WebServer/36]    [L0] core.server.common_services.auth_server:: password mismatch SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:18:52.819 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv auth_api.create_auth reqid wss://noobaa-mgmt.openshift-storage.svc.cluster.local:443/rpc/-1658 connid ws://[::ffff:10.130.0.148]:39970/(2lkyyis) Error: credentials not found
    at /root/node_modules/noobaa-core/src/server/common_services/auth_server.js:71:39
    at async /root/node_modules/noobaa-core/src/rpc/rpc.js:340:32
    at async RPC._on_request (/root/node_modules/noobaa-core/src/rpc/rpc.js:348:25)
Jul-8 5:18:56.203 [WebServer/36]    [L0] core.server.common_services.auth_server:: credentials account not found SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:18:56.247 [BGWorkers/32]    [L0] core.server.bg_services.md_aggregator:: MD aggregator start running
Jul-8 5:18:56.248 [BGWorkers/32]    [L0] core.server.bg_services.md_aggregator:: find_next_range: from_time 1625721412532 till_time* 33715 target_now* 33715 global_last_update 1625721412532
Jul-8 5:18:56.248 [BGWorkers/32]    [L0] core.server.bg_services.md_aggregator:: find_next_range:: next work from_time 1625721412532 till_time* 30000 target_now* 33715
Jul-8 5:18:56.302 [WebServer/36]    [L0] core.server.common_services.auth_server:: password mismatch SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:18:56.302 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv auth_api.create_auth reqid wss://noobaa-mgmt.openshift-storage.svc.cluster.local:443/rpc/-1661 connid ws://[::ffff:10.130.0.148]:39970/(2lkyyis) Error: credentials not found
    at /root/node_modules/noobaa-core/src/server/common_services/auth_server.js:71:39
    at async /root/node_modules/noobaa-core/src/rpc/rpc.js:340:32
    at async RPC._on_request (/root/node_modules/noobaa-core/src/rpc/rpc.js:348:25)
Jul-8 5:18:57.256 [BGWorkers/32]    [L0] core.server.bg_services.cluster_hb:: no local cluster info. HB is not written
Jul-8 5:18:57.381 [BGWorkers/32]   [LOG] CONSOLE:: TieringSpillbackWorker: start running
Jul-8 5:18:58.635 [BGWorkers/32]    [L0] core.server.bg_services.cluster_master:: no local cluster info or server is not part of a cluster. therefore will be cluster master
Jul-8 5:18:58.638 [WebServer/36]    [L0] core.server.system_services.redirector:: publish_to_cluster: server_inter_process update_master_change { is_master: true } [ 'fcall://fcall(2lq5x8u)', 'ws://[::ffff:127.0.0.1]:44874/(4dep0fq)', 'ws://[::ffff:127.0.0.1]:44962/(4ouj6fn)' ]
Jul-8 5:18:59.444 [BGWorkers/32]    [L0] core.server.bg_services.agent_blocks_reclaimer:: AGENT_BLOCKS_RECLAIMER: BEGIN
Jul-8 5:18:59.453 [WebServer/36]    [L0] core.server.common_services.auth_server:: credentials account not found SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:18:59.551 [WebServer/36]    [L0] core.server.common_services.auth_server:: password mismatch SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:18:59.552 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv auth_api.create_auth reqid wss://noobaa-mgmt.openshift-storage.svc.cluster.local:443/rpc/-1664 connid ws://[::ffff:10.130.0.148]:39970/(2lkyyis) Error: credentials not found
    at /root/node_modules/noobaa-core/src/server/common_services/auth_server.js:71:39
    at async /root/node_modules/noobaa-core/src/rpc/rpc.js:340:32
    at async RPC._on_request (/root/node_modules/noobaa-core/src/rpc/rpc.js:348:25)
Jul-8 5:19:01.698 [BGWorkers/32]    [L0] core.server.system_services.system_store:: SystemStore.make_changes: {
  update: {
    buckets: [ { _id: 60e52c5863d9a8002a621eec, storage_stats: { chunks_capacity: 0, objects_size: 0, objects_count: 0, stats_by_content_type: [], blocks_size: 0, pools: {}, objects_hist: [], last_update: 1625721442532 }, dont_change_last_update: true } ],
    pools: [ { _id: 60e52c5863d9a8002a621ee6, storage_stats: { blocks_size: 0, last_update: 1625721442532 }, dont_change_last_update: true } ]
  }
}
Jul-8 5:19:01.706 [BGWorkers/32]    [L0] core.server.system_services.system_store:: SystemStore.make_changes: {
  update: { systems: [ { _id: 60e52c5863d9a8002a621ee4, global_last_update: 1625721442532 } ] }
}
Jul-8 5:19:01.713 [WebServer/36]    [L0] core.server.system_services.redirector:: publish_to_cluster: server_inter_process load_system_store { since: 1625721541706 } [ 'fcall://fcall(2lq5x8u)', 'ws://[::ffff:127.0.0.1]:44874/(4dep0fq)', 'ws://[::ffff:127.0.0.1]:44962/(4ouj6fn)' ]
Jul-8 5:19:02.718 [WebServer/36]    [L0] core.server.common_services.auth_server:: credentials account not found SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:19:02.816 [WebServer/36]    [L0] core.server.common_services.auth_server:: password mismatch SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:19:02.816 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv auth_api.create_auth reqid wss://noobaa-mgmt.openshift-storage.svc.cluster.local:443/rpc/-1667 connid ws://[::ffff:10.130.0.148]:39970/(2lkyyis) Error: credentials not found
    at /root/node_modules/noobaa-core/src/server/common_services/auth_server.js:71:39
    at async /root/node_modules/noobaa-core/src/rpc/rpc.js:340:32
    at async RPC._on_request (/root/node_modules/noobaa-core/src/rpc/rpc.js:348:25)
Jul-8 5:19:03.273 [BGWorkers/32]    [L0] core.server.bg_services.md_aggregator:: find_next_range: from_time 1625721442532 till_time* 3715 target_now* 3715 global_last_update 1625721412532
Jul-8 5:19:03.273 [BGWorkers/32]    [L0] core.server.bg_services.md_aggregator:: find_next_range:: next work from_time 1625721442532 till_time* 3715 target_now* 3715
Jul-8 5:19:03.284 [BGWorkers/32]    [L0] core.server.system_services.system_store:: SystemStore.make_changes: {
  update: {
    buckets: [ { _id: 60e52c5863d9a8002a621eec, storage_stats: { chunks_capacity: 0, objects_size: 0, objects_count: 0, stats_by_content_type: [], blocks_size: 0, pools: {}, objects_hist: [], last_update: 1625721446247 }, dont_change_last_update: true } ],
    pools: [ { _id: 60e52c5863d9a8002a621ee6, storage_stats: { blocks_size: 0, last_update: 1625721446247 }, dont_change_last_update: true } ]
  }
}
Jul-8 5:19:03.293 [BGWorkers/32]    [L0] core.server.system_services.system_store:: SystemStore.make_changes: {
  update: { systems: [ { _id: 60e52c5863d9a8002a621ee4, global_last_update: 1625721446247 } ] }
}
Jul-8 5:19:03.299 [WebServer/36]    [L0] core.server.system_services.redirector:: publish_to_cluster: server_inter_process load_system_store { since: 1625721543293 } [ 'fcall://fcall(2lq5x8u)', 'ws://[::ffff:127.0.0.1]:44874/(4dep0fq)', 'ws://[::ffff:127.0.0.1]:44962/(4ouj6fn)' ]
Jul-8 5:19:04.325 [BGWorkers/32]    [L0] core.server.bg_services.md_aggregator:: find_next_range: from_time 1625721446247 till_time* 0 target_now* 0 global_last_update 1625721412532
Jul-8 5:19:04.325 [BGWorkers/32]    [L0] core.server.bg_services.md_aggregator:: find_next_range:: no more work from_time 1625721446247 till_time* 0 target_now* 0
Jul-8 5:19:04.325 [BGWorkers/32]    [L0] core.server.bg_services.md_aggregator:: MD aggregator done
Jul-8 5:19:05.982 [WebServer/36]    [L0] core.server.common_services.auth_server:: credentials account not found SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:19:06.083 [WebServer/36]    [L0] core.server.common_services.auth_server:: password mismatch SENSITIVE-9cf0fd89409efef8 noobaa
Jul-8 5:19:06.083 [WebServer/36] [ERROR] CONSOLE:: RPC._on_request: ERROR srv auth_api.create_auth reqid wss://noobaa-mgmt.openshift-storage.svc.cluster.local:443/rpc/-1670 connid ws://[::ffff:10.130.0.148]:39970/(2lkyyis) Error: credentials not found
    at /root/node_modules/noobaa-core/src/server/common_services/auth_server.js:71:39
    at async /root/node_modules/noobaa-core/src/rpc/rpc.js:340:32
    at async RPC._on_request (/root/node_modules/noobaa-core/src/rpc/rpc.js:348:25)

On the console status show cannot create an auth token for admin, error: credentials not found The status on the yaml shows: 

status:
  accounts:
    admin:
      secretRef:
        name: noobaa-admin
        namespace: openshift-storage
  actualImage: >-
    registry.redhat.io/ocs4/mcg-core-rhel8@sha256:6ff8645efdde95fa97d496084d3555b7680895f0b79c147f2a880b43742af3a4
  conditions:
    - lastHeartbeatTime: '2021-07-08T04:47:41Z'
      lastTransitionTime: '2021-07-08T04:47:41Z'
      message: 'cannot create an auth token for admin, error: credentials not found'
      reason: TemporaryError
      status: 'False'
      type: Available
    - lastHeartbeatTime: '2021-07-08T04:47:41Z'
      lastTransitionTime: '2021-07-08T04:47:41Z'
      message: 'cannot create an auth token for admin, error: credentials not found'
      reason: TemporaryError
      status: 'True'
      type: Progressing
    - lastHeartbeatTime: '2021-07-08T04:47:41Z'
      lastTransitionTime: '2021-07-08T04:47:41Z'
      message: 'cannot create an auth token for admin, error: credentials not found'
      reason: TemporaryError
      status: 'False'
      type: Degraded
    - lastHeartbeatTime: '2021-07-08T04:47:41Z'
      lastTransitionTime: '2021-07-08T04:47:41Z'
      message: 'cannot create an auth token for admin, error: credentials not found'
      reason: TemporaryError
      status: 'False'
      type: Upgradeable
  observedGeneration: 2
  phase: Configuring
  readme: "\n\n\tNooBaa operator is still working to reconcile this system.\n\tCheck out the system status.phase, status.conditions, and events with:\n\n\t\tkubectl -n openshift-storage describe noobaa\n\t\tkubectl -n openshift-storage get noobaa -o yaml\n\t\tkubectl -n openshift-storage get events --sort-by=metadata.creationTimestamp\n\n\tYou can wait for a specific condition with:\n\n\t\tkubectl -n openshift-storage wait noobaa/noobaa --for condition=available --timeout -1s\n\n\tNooBaa Core Version:     5.7.0\n\tNooBaa Operator Version: 5.7.0\n"

I can see the noobaa-admin secret created in the namespace/project.

Need help in finding out how can this problem be fixed.

jackyalbo commented 3 years ago

Does the secret have the email and password set in it? Can you also attach the noobaa-operator logs?

m-usmanayub commented 3 years ago

Yes I can see the email and password set in the secret. noobaa-operator logs are pasted below

time="2021-07-11T08:29:47Z" level=info msg="Start ..." sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="✅ Exists: NooBaa \"noobaa\"\n"
time="2021-07-11T08:29:47Z" level=info msg="Memory Usage: Phase \"Starting\" - Alloc = 56 MiB  Sys = 146 MiB  NumGC = 21858" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="SetPhase: \"Verifying\"" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="Parsed image (Named) registry.redhat.io/ocs4/mcg-core-rhel8@sha256:6ff8645efdde95fa97d496084d3555b7680895f0b79c147f2a880b43742af3a4" func=CheckSystemCR sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="Using custom image name \"registry.redhat.io/ocs4/mcg-core-rhel8@sha256:6ff8645efdde95fa97d496084d3555b7680895f0b79c147f2a880b43742af3a4\" the default is \"noobaa/noobaa-core\"" func=CheckSystemCR sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="SetPhase: \"Creating\"" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="ReconcileObject: Done - unchanged ServiceAccount /api/v1/namespaces/openshift-storage/serviceaccounts/noobaa" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="ReconcileObject: Done - unchanged Secret /api/v1/namespaces/openshift-storage/secrets/noobaa-server" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="ReconcileObject: Done - unchanged Secret /api/v1/namespaces/openshift-storage/secrets/noobaa-db" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="ReconcileObject: Done - unchanged Secret /api/v1/namespaces/openshift-storage/secrets/noobaa-root-master-key" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="✅ Already Exists: ConfigMap \"noobaa-postgres-config\"\n"
time="2021-07-11T08:29:47Z" level=info msg="ReconcileObject: Done - unchanged StatefulSet /apis/apps/v1/namespaces/openshift-storage/statefulsets/noobaa-db-pg" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="ReconcileObject: Done - unchanged Service /api/v1/namespaces/openshift-storage/services/noobaa-db-pg" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="❌ Not Found: Service \"noobaa-db\"\n"
time="2021-07-11T08:29:47Z" level=info msg="ReconcileObject: Done - unchanged Service /api/v1/namespaces/openshift-storage/services/noobaa-mgmt" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="ReconcileObject: Done - unchanged StatefulSet /apis/apps/v1/namespaces/openshift-storage/statefulsets/noobaa-core" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="ReconcileObject: Done - unchanged Route " sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="ReconcileObject: Done - unchanged Service /api/v1/namespaces/openshift-storage/services/s3" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="ReconcileObject: Done - unchanged Route " sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="Reconciling Backing Store Credentials" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="Not running in AWS. will attempt to create a ceph objectstore user" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="❌ Not Found: CephObjectStoreUser \"noobaa-ceph-objectstore-user\"\n"
time="2021-07-11T08:29:47Z" level=info msg="did not find any ceph objectstore to use as backing store" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="SetPhase: \"Connecting\"" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="Collected addresses: &{NodePorts:[https://10.150.8.166:31268] PodPorts:[https://10.128.2.34:8443] InternalIP:[https://172.30.31.59:443] InternalDNS:[https://noobaa-mgmt.openshift-storage.svc:443] ExternalIP:[] ExternalDNS:[https://noobaa-mgmt-openshift-storage.apps.y000dlvocp01.removed]}" func=CheckServiceStatus service=noobaa-mgmt sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="✈️  RPC: auth.read_auth() Request: <nil>"
time="2021-07-11T08:29:47Z" level=info msg="✅ RPC: auth.read_auth() Response OK: took 0.1ms"
time="2021-07-11T08:29:47Z" level=info msg="Collected addresses: &{NodePorts:[] PodPorts:[] InternalIP:[https://172.30.52.182:443] InternalDNS:[https://s3.openshift-storage.svc:443] ExternalIP:[] ExternalDNS:[https://s3-openshift-storage.apps.removed}" func=CheckServiceStatus service=s3 sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="SetPhase: \"Configuring\"" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="ReconcileObject: Done - updated Secret /api/v1/namespaces/openshift-storage/secrets/noobaa-admin" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="✈️  RPC: system.get_system_status() Request: <nil>"
time="2021-07-11T08:29:47Z" level=info msg="✅ RPC: system.get_system_status() Response OK: took 0.1ms"
time="2021-07-11T08:29:47Z" level=info msg="✈️  RPC: auth.create_auth() Request: {System:noobaa Role:admin Email:admin@noobaa.io Password:Z7BiAnmlbgu9u+baTVkHKQ==}"
time="2021-07-11T08:29:47Z" level=error msg="⚠️  RPC: auth.create_auth() Response Error: Code=UNAUTHORIZED Message=credentials not found"
time="2021-07-11T08:29:47Z" level=error msg="ReconcileObject: Error Secret  cannot create an auth token for admin, error: credentials not found" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="SetPhase: temporary error during phase \"Configuring\"" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=warning msg="⏳ Temporary Error: cannot create an auth token for admin, error: credentials not found" sys=openshift-storage/noobaa
time="2021-07-11T08:29:47Z" level=info msg="UpdateStatus: Done generation 2" sys=openshift-storage/noobaa

same logs are repeating continuously.

renich commented 1 year ago

ping