gitCredentialsSecret doesn't follow RFC 1123 specification

[petardjurkovic]

Affected Stackable version

24.7.0

Affected Apache Airflow version

2.9.2

Current and expected behavior

When gitSync is enabled

dagsGitSync:
      - repo: https://gitlink/airflow-dags.git
        branch: release/test
        gitFolder: ./dags
        credentialsSecret: git-credentials
      - repo: https://gitlink/airflow-plugins.git
        branch: release/test
        gitFolder: ./plugins
        credentialsSecret: git-credentials

--- apiVersion: v1 kind: Secret metadata: name: git-credentials type: Opaque stringData: user: someuser password: somepassword down errors preventing airflow-webserver to be created: create Pod airflow-webserver-default-1 in StatefulSet airflow-webserver-default failed error: Pod "airflow-webserver-default-1" is invalid: [spec.containers[2].env[2].valueFrom.secretKeyRef.name: Invalid value: "gitCredentialsSecret": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.containers[2].env[3].valueFrom.secretKeyRef.name: Invalid value: "gitCredentialsSecret": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')]

Possible solution

No response

Additional context

No response

Environment

No response

Would you like to work on fixing this bug?

maybe

[adwk67]

Cause: the secret is not parsed correctly. This was not apparent in release-24.3 as the env-variables were not passed through consistently to git-sync. See https://github.com/stackabletech/airflow-operator/pull/456.

[petardjurkovic]

Cause: the secret is not parsed correctly. This was not apparent in release-24.3 as the env-variables were not passed through consistently to git-sync. See #456.

Hi @adwk67 do you suggest me to downgrade operator to 24.3 version?

[adwk67]

@petardjurkovic Well, the required env-vars for the secret were passed through incorrectly in 24.3, so I'm testing my fix just now. BTW, I see in your example you have multiple gitsyncs, although at the moment only one is supported (see the IMPORTANT note at the bottom).

[petardjurkovic]

thanks @adwk67 on the fast response, we will reorganize airflow dags/plugins in one repo.

[PaulienVa]

@adwk67 we tested the nightly and keep on getting this bug so it does not seem to be solved. The only way we have to implement the gitsync is to do a podOverrides with an environment variable for "GITSYNC_PASSWORD". Is it possible to still get it fixed?

[adwk67]

@PaulienVa Could you open a new issue with a specific example yaml that we can use to try and reproduce this? We have an integration test for this that runs successfully so maybe there is a use-case that we have not foreseen or covered.

[PaulienVa]

@adwk67 just did: https://github.com/stackabletech/airflow-operator/issues/500

[lfrancke]

@adwk67 Should this appear in the release notes for 24.11? And if so: Could you write a short sentence for it.

[adwk67]

Ah, this was covered here, not linked to this issue for some reason.

[lfrancke]

Sorry, I had already forgotten about that :(