Databases not being onboarded while deploying on GKE

[masterlittle]

Affected module Does it impact the UI, backend or Ingestion Framework?

Ingestion

Describe the bug A clear and concise description of what the bug is.

I have deployed Openmetadata 1.1 on GKE cluster. When I'm adding a bigquery or postgres database, and click on Test connection getting the error - Failed to deploy pipeline [test-connection-BigQuery-Kp3J2ByY] due to [HTTP/1.1 header parser received no bytes].

On checking the logs this api - http://d.d.in/api/v1/automations/workflows/trigger/5d444d5a-2673-4455-90ad-551b0ff595e6 is giving a 400 response.

When trying to ingest something, getting an import error -

Broken DAG: [/airflow-dags/dags/bigquery_metadata_D2a7xUUC.py] Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.9/site-packages/metadata/ingestion/ometa/ometa_api.py", line 590, in get_by_name
    return self._get(
  File "/home/airflow/.local/lib/python3.9/site-packages/metadata/ingestion/ometa/ometa_api.py", line 620, in _get
    raise EmptyPayloadException(
metadata.ingestion.ometa.ometa_api.EmptyPayloadException: Got an empty response when trying to GET from /services/databaseServices/name/bigquery

Version:

• OS: [e.g. iOS] - Linux
• Python version:
• OpenMetadata version: [e.g. 0.8] - 1.1
• OpenMetadata Ingestion package version: [e.g. openmetadata-ingestion[docker]==XYZ] latest

Additional context I'm using the inbuilt openmetadata airflow here.

Values.yml

replicaCount: 1
service:
  type: LoadBalancer
  port: 80
  adminPort: 8586
  annotations:
    networking.gke.io/load-balancer-type: "Internal"
extraVolumes:
- name: openmetadata-jwt-vol
  secret:
    secretName: openmetadata-jwt-keys
extraVolumeMounts:
- name: openmetadata-jwt-vol
  mountPath: "/etc/openmetadata/jwtkeys"
  readOnly: true
openmetadata:
  config:
    # Values can be OFF, ERROR, WARN, INFO, DEBUG, TRACE, or ALL
    logLevel: DEBUG
    clusterName: openmetadata
    maskPasswordsApi: false
    openmetadata:
      host: openmetadata
      # URI to use with OpenMetadata Alerts Integrations
      uri: "http://openmetadata:8585"
      port: 8585
      adminPort: 8586
    elasticsearch:
      host: elasticsearch
      searchType: elasticsearch
      port: 9200
      scheme: http
      connectionTimeoutSecs: 5
      socketTimeoutSecs: 60
      batchSize: 10
      searchIndexMappingLanguage: "EN"
      keepAliveTimeoutSecs: 600
      trustStore:
        enabled: false
        path: ""
        password:
          secretRef: "elasticsearch-truststore-secrets"
          secretKey: "openmetadata-elasticsearch-truststore-password"
      auth:
        enabled: false
        username: "elasticsearch"
        password:
          secretRef: elasticsearch-secrets
          secretKey: openmetadata-elasticsearch-password
    database:
      host: 1xxxxx
      port: 5432
      driverClass: org.postgresql.Driver
      dbScheme: postgresql
      dbUseSSL: false
      databaseName: openmetadata
      auth:
        username: openmetadata_user
        password:
          secretRef: postgres-secrets
          secretKey: openmetadata-postgres-password
    pipelineServiceClientConfig:
      enabled: true
      className: "org.openmetadata.service.clients.pipeline.airflow.AirflowRESTClient"
      # endpoint url for airflow
      apiEndpoint: http://xxxxx:80
      # this will be the api endpoint url of OpenMetadata Server
      metadataApiEndpoint: http://d.d.in/api
      # possible values are "no-ssl", "ignore", "validate"
      verifySsl: "no-ssl"
      hostIp: "d.d.in"
      ingestionIpInfoEnabled: true
      # healthCheckInterval in seconds
      healthCheckInterval: 300
      # local path in Airflow Pod
      sslCertificatePath: "/no/path"
      auth:
        username: admin
        password:
          secretRef: airflow-secrets
          secretKey: openmetadata-airflow-password
    authorizer:
      className: "org.openmetadata.service.security.DefaultAuthorizer"
      containerRequestFilter: "org.openmetadata.service.security.JwtFilter"
      enforcePrincipalDomain: true
      initialAdmins:
        - xxx
      principalDomain: "d.in"
    authentication:
      provider: "google"
      publicKeys:
        - "https://www.googleapis.com/oauth2/v3/certs"
        - "http://openmetadata:8585/api/v1/system/config/jwks"
        - "http://d.d.in/api/v1/system/config/jwks"
      authority: "https://accounts.google.com"
      clientId: "xxxgoogleusercontent.com"
      callbackUrl: "http://d.din/callback"
    smtpConfig:
      enableSmtpServer: true
      emailingEntity: "OpenMetadata"
      supportUrl: "https://d.slack.com"
      transportationStrategy: "SMTP_TLS"
      openMetadataUrl: "http://d.d.in"
      serverEndpoint: "email-smtp.us-east-1.amazonaws.com"
      serverPort: "587"
      senderMail: "dataengineering@d.in"
      username: "xxxxx"
      password:
        secretRef: "smtp-secrets"
        secretKey: "openmetadata-smtp-password"
    jwtTokenConfiguration:
      rsapublicKeyFilePath: "/etc/openmetadata/jwtkeys/public_key.der"
      rsaprivateKeyFilePath: "/etc/openmetadata/jwtkeys/private_key.der"
      jwtissuer: "d.d.in" # update this as per your environment
      keyId: "xxxxxx" # update this to a unique uuid4

[akash-jain-10]

Hello @masterlittle - Can you please update to our latest release and verify if the issue is still persistent ? Thanks.

[akash-jain-10]

Closing the issue here. If needed further discussions, please join our community slack and we can have further conversations there! Thanks.