Deploying to local Kubernetes via Helm: Airbyte-Temporal fails to initialize: Error: unable to list search attributes: Namespace default is not found.

john-motif commented 1 year ago

What method are you using to run Airbyte?

Docker

Platform Version or Helm Chart Version

0.50.33

What step the error happened?

None

Revelant information

I've been attempting to connect to an external Supabase DB locally before rolling on a docker kubernetes cluster out this deployment to production (per these instructions and these).

It seems that when airbyte-temporal attempts to startup, it fails to correctly create the desired namespace:

Subsequently, Airbyte's other components will start and I can access the UI, but any attempt to create sources/destinations causes context-deadline exceeded errors in airbyte-temporal. I suspect it's not initialized properly/is hanging indefinitely.

Any recommendations on related fixes or further investigation would be hugely appreciated!

my values.yaml:

postgresql:
  enabled: false
  ## image.repository Repository for airbyte-db statefulset
  image:
    repository: airbyte/db
  postgresqlUsername: airbyte
  postgresqlPassword: airbyte
  postgresqlDatabase: db-airbyte
  existingSecret: ""
  containerSecurityContext:
    runAsNonRoot: true
  commonAnnotations:
    helm.sh/hook: pre-install,pre-upgrade
    helm.sh/hook-weight: "-1"
externalDatabase: 
  host: <REDACTED DB HOST>
  user: "postgres"
  password: <REDACTED PW>
  database: "postgres"
  port: 5432
  jdbcUrl: "jdbc:postgresql://<REDACTED DB HOST>.co:5432/postgres"

Relevant log output

2023-11-07 11:19:32 Temporal server started.
2023-11-07 11:19:32 Registering default namespace: default.
2023-11-07 11:19:32 + echo 'Temporal server started.'
2023-11-07 11:19:32 + [[ false != true ]]
2023-11-07 11:19:32 + register_default_namespace
2023-11-07 11:19:32 + echo 'Registering default namespace: default.'
2023-11-07 11:19:32 + temporal operator namespace describe default
2023-11-07 11:19:32 {"level":"info","ts":"2023-11-07T19:19:32.624Z","msg":"Current reachable members","service":"history","component":"service-resolver","service":"frontend","addresses":["10.1.0.234:7233"],"logging-call-at":"rpServiceResolver.go:284"}
....
2023-11-07 11:19:33 Error: Namespace default is not found.
2023-11-07 11:19:33 ('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)
2023-11-07 11:19:33 Default namespace default not found. Creating...
2023-11-07 11:19:33 + echo 'Default namespace default not found. Creating...'
2023-11-07 11:19:33 + temporal operator namespace create --retention 1 --description 'Default namespace for Temporal Server.' default

john-motif commented 6 months ago

Unfortunately no, I never found a solution, however my issue did not occur spontaneously but rather while attempting to modify the helm configuration.

On Sat, Apr 20, 2024 at 8:57 AM Praer (Suthira Owlarn) < @.***> wrote:

Hi @john-motif https://github.com/john-motif, thank you for submitting this issue. Did you find a solution in the end? I'm seeing similar errors running in EKS/kubernetes (chart version 0.63.18). It seemed like all of our syncs stopped working without any reason..

— Reply to this email directly, view it on GitHub https://github.com/airbytehq/airbyte/issues/32267#issuecomment-2067714861, or unsubscribe https://github.com/notifications/unsubscribe-auth/A5WZRHAX6J37ZSODPWSN3TLY6KF5VAVCNFSM6AAAAAA7BXC4L2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRXG4YTIOBWGE . You are receiving this because you were mentioned.Message ID: @.***>

-- John

octavia-squidington-iii commented 1 day ago

At Airbyte, we seek to be clear about the project priorities and roadmap. This issue has not had any activity for 180 days, suggesting that it's not as critical as others. It's possible it has already been fixed. It is being marked as stale and will be closed in 20 days if there is no activity. To keep it open, please comment to let us know why it is important to you and if it is still reproducible on recent versions of Airbyte.

airbytehq / airbyte