[knox] redirect backports repo on oldoldstable

[Abinash-Sharma]

The knox.sh script has been updated to ensure compatibility across various Dataproc image versions. Here's a summary of the changes:

2.0.X-debian10: Addressed an issue where the script failed for image versions less than or equal to 2.0.102-debian10 due to original keys had expired at https://keyserver.ubuntu.com. Note: Cluster creation with these versions requires the cloud-platform scope. 2.1.X-debian11: The script functioned correctly with all 2.1.X-debian11 image versions. 2.2.x-debian12: Script was not working for 2.2 images . Resolved a problem caused by the hostname returning an FQDN in 2.2.x-debian12 images.

Testing:

Thorough testing was conducted with various cluster naming conventions (e.g., proxy-cluster, proxy-cluster-x, proxy-cluster-x-v) and across all supported image versions.

[cjac]

Abinash, I'm pretty swamped with a couple of other issues. Can you review with @prince-cs to make sure all of the tests are passing?

[cjac]

/gcbrun

[Abinash-Sharma]

My testing : knox.sh script is working fine all images in Debian and Ubuntu Dataproc Image. Script need modification for Rocky Images (Nothing to do with this PR) .

[prince-cs]

The following error was faced for Ubuntu 2.2 image. Just to confirm I will run the presubmit tests once again. All the other versions are passing the tests, only 2.2 ubuntu image has the failure

++ /usr/share/google/get_metadata_value attributes/knox-gw-config
+ readonly KNOX_GW_CONFIG_GCS=gs://dataproc-init-actions-test-cloud-dataproc-ci/20241008-033042-z80w/knox
+ KNOX_GW_CONFIG_GCS=gs://dataproc-init-actions-test-cloud-dataproc-ci/20241008-033042-z80w/knox
++ sudo -u knox mktemp -d -t knox-init-action-config-XXXX
+ readonly KNOX_GW_CONFIG=/tmp/knox-init-action-config-0H5i
+ KNOX_GW_CONFIG=/tmp/knox-init-action-config-0H5i
+ readonly KNOX_HOME=/usr/lib/knox
+ KNOX_HOME=/usr/lib/knox
+ has_gateway_config_changed=
+ [[ 0 -gt 0 ]]
+ install
+ local master_name
++ /usr/share/google/get_metadata_value attributes/dataproc-master
+ master_name=test-knox-ha-2-2-20241008-033131-sgr8-m-0
++ hostname -s
+ [[ test-knox-ha-2-2-20241008-033131-sgr8-m-0 == test-knox-ha-2-2-20241008-033131-sgr8-m-0 ]]
+ [[ -z gs://dataproc-init-actions-test-cloud-dataproc-ci/20241008-033042-z80w/knox ]]
+ remove_old_backports
+ is_debian12
+ is_debian
++ os_id
++ grep '^ID=' /etc/os-release
++ cut -d= -f2
++ xargs
+ [[ ubuntu == \d\e\b\i\a\n ]]
+ is_debian11
+ is_debian
++ os_id
++ grep '^ID=' /etc/os-release
++ xargs
++ cut -d= -f2
+ [[ ubuntu == \d\e\b\i\a\n ]]
+ debdists=https://deb.debian.org/debian/dists
++ curl -s https://deb.debian.org/debian/dists/oldoldstable/Release
++ awk '/^Codename/ {print $2}'
+ oldoldstable=buster
++ curl -s https://deb.debian.org/debian/dists/oldstable/Release
++ awk '/^Codename/ {print $2}'
+ oldstable=bullseye
++ awk '/^Codename/ {print $2}'
++ curl -s https://deb.debian.org/debian/dists/stable/Release
+ stable=bookworm
+ matched_files=($(test -d /etc/apt && grep -rsil '\-backports' /etc/apt/sources.list*||:))
++ test -d /etc/apt
++ grep -rsil '\-backports' /etc/apt/sources.list /etc/apt/sources.list.d
+ for filename in "${matched_files[@]}"
+ perl -pi -e 's{^(deb[^\s]*) https?://[^/]+/debian buster-backports }
                  {$1 https://archive.debian.org/debian buster-backports }g' /etc/apt/sources.list
+ sed -i 's/^deb*/#deb/g' /etc/apt/sources.list.d/google-cloud-logging.list
sed: can't read /etc/apt/sources.list.d/google-cloud-logging.list: No such file or directory

[prince-cs]

/gcbrun

[prince-cs]

/gcbrun

[prince-cs]

So, the latest run also reveals the failure in 2.2 ubuntu image. I have attached the error logs for your reference. My guess is that we should just skip the remove_old_backports for ubnutu22

[cjac]

Remove the sed line. The Perl line before does what needs doing.

• sed -i 's/^deb*/#deb/g' /etc/apt/sources.list.d/google-cloud-logging.list sed: can't read /etc/apt/sources.list.d/google-cloud-logging.list: No such file or directory

[cjac]

/gcbrun