stackabletech / docker-images

Apache License 2.0
17 stars 4 forks source link

chore(spark): Update container images ahead of Stackable Release 24.11 #838

Closed NickLarsenNZ closed 1 month ago

NickLarsenNZ commented 1 month ago

Part of #831.

Add 3.5.2 (check dependency versions, compared to 3.5.2). Remove 3.4.2, 3.4.3.

[!TIP] Please add the scheduled-for/20XX-XX label.

### Update tasks
- [x] Update `versions.py` to reflect the agreed upon versions in the spreadsheet (including the removal of old versions).
- [x] Upload new version (see `spark/upload_new_spark_version.sh`).
- [x] Update `versions.py` to the latest supported version of JVM (base and devel).
- [x] Update other dependencies if applicable (eg: python, jmx_exporter, aws_java_sdk_bundle, etc).
- [x] Check other operators (getting_started / kuttl) for usage of the versions. Add the PR(s) to the list below.
- [x] Update the version in demos. Add the PR(s) to the list below.
### Related Pull Requests
- [ ] https://github.com/stackabletech/docker-images/pull/848
- [ ] https://github.com/stackabletech/spark-k8s-operator/pull/459
- [x] https://github.com/stackabletech/airflow-operator/pull/504 [_Link to any other operator PRs (getting_started / kuttl)_]
- [x] Updated dags repo https://github.com/stackable-airflow/dags/commit/a82079cdc09ce17f422ac38d9c4b1c2f67a8d391
- [x] _Link to demo PR (raise against the `next` branch)_ https://github.com/stackabletech/demos/pull/96
- [ ] https://github.com/stackabletech/spark-k8s-operator/pull/461
### Acceptance
- [x] Can build image locally
- [x] Kuttl smoke tests passes locally
- [ ] Release notes written in a comment below
- [ ] Applicable `release-note` label added to this issue
Testing instructions ```shell # See the latest version at https://pypi.org/project/image-tools-stackabletech/ pip install image-tools-stackabletech==0.0.12 bake --product spark-k8s=x.y.z # where x.y.z is the new version added in this PR kind load docker-image docker.stackable.tech/stackable/spark-k8s:x.y.z-stackable0.0.0-dev # Change directory into the spark-k8s-operator repository and update the # product version in tests/test-definition.yaml ./scripts/run-tests --test-suite smoke-latest # or similar ```

Please consider updating this template if these instructions are wrong, or could be made clearer.

lfrancke commented 1 month ago

Is there anything left to review for this issue? It looks like all PRs have been merged already.

razvan commented 1 month ago

I added a new PR: https://github.com/stackabletech/spark-k8s-operator/pull/461

And I'm not sure what to do with the last two unchecked items

lfrancke commented 1 month ago

The last two items are for the Acceptance phase and in addition do not apply here :)