stackabletech / spark-k8s-operator

Operator for Apache Spark-on-Kubernetes for Stackable Data Platform

https://stackable.tech

Other

51 stars 2 forks source link

[Merged by Bors] - Fix/custom s3 certificates #247

Closed Maleware closed 1 year ago

Maleware commented 1 year ago

Description

This PR is supposed to enable TLS for spark-k8s for S3 buckets. Fixes #194 TODO: add s3 certificate for history server access.

Definition of Done Checklist

Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
Please make sure all these things are done and tick the boxes

# Author
- [x] Changes are OpenShift compatible
- [x] Helm chart can be installed and deployed operator works
- [x] Integration tests passed (for non trivial changes)

# Reviewer
- [ ] Code contains useful comments
- [x] (Integration-)Test cases added
- [x] Documentation added or updated
- [ ] Changelog updated
- [x] Cargo.toml only contains references to git tags (not specific commits or branches)

# Acceptance
- [ ] Feature Tracker has been updated
- [ ] Proper release label has been added

Once the review is done, comment bors r+ (or bors merge) to merge. Further information

adwk67 commented 1 year ago

Openshift tests:

--- PASS: kuttl (1380.65s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/spark-pi-public-s3_openshift-true_spark-3.3.0-stackable0.0.0-dev (88.03s)
        --- PASS: kuttl/harness/spark-pi-private-s3_openshift-true_spark-3.3.0-stackable0.0.0-dev (100.78s)
        --- PASS: kuttl/harness/spark-examples_openshift-true_spark-3.4.0-stackable0.0.0-dev (48.32s)
        --- PASS: kuttl/harness/spark-examples_openshift-true_spark-3.3.0-stackable0.0.0-dev (52.71s)
        --- PASS: kuttl/harness/spark-ny-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev_s3-use-tls-false (109.41s)
        --- PASS: kuttl/harness/spark-ny-public-s3_openshift-true_spark-3.3.0-stackable0.0.0-dev_s3-use-tls-false (120.17s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3-image_openshift-true_spark-3.4.0-stackable0.0.0-dev_ny-tlc-report-0.1.0 (115.74s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3-image_openshift-true_spark-3.3.0-stackable0.0.0-dev_ny-tlc-report-0.1.0 (124.04s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev (120.29s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3_openshift-true_spark-3.3.0-stackable0.0.0-dev (117.92s)
        --- PASS: kuttl/harness/resources_openshift-true_spark-3.4.0-stackable0.0.0-dev (154.66s)
        --- PASS: kuttl/harness/resources_openshift-true_spark-3.3.0-stackable0.0.0-dev (201.79s)
        --- PASS: kuttl/harness/spark-history-server_openshift-true_spark-3.4.0-stackable0.0.0-dev (230.03s)
        --- PASS: kuttl/harness/spark-history-server_openshift-true_spark-3.3.0-stackable0.0.0-dev (208.98s)
        --- PASS: kuttl/harness/spark-pi-private-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev (100.56s)
        --- PASS: kuttl/harness/logging_openshift-true_spark-3.3.0-stackable0.0.0-dev_ny-tlc-report-0.1.0 (352.51s)
        --- PASS: kuttl/harness/logging_openshift-true_spark-3.4.0-stackable0.0.0-dev_ny-tlc-report-0.1.0 (349.18s)
        --- PASS: kuttl/harness/spark-pi-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev (89.77s)

adwk67 commented 1 year ago

Jenkins tests: :green_circle: https://ci.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/spark-k8s-operator-it-custom/88/

adwk67 commented 1 year ago

Jenkins run (updated): :green_circle: https://ci.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/spark-k8s-operator-it-custom/89/

adwk67 commented 1 year ago

Openshift tests:

--- PASS: kuttl (1815.94s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3-image_openshift-true_spark-3.3.0-stackable0.0.0-dev_ny-tlc-report-0.1.0 (122.83s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev (141.16s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3_openshift-true_spark-3.3.0-stackable0.0.0-dev (109.38s)
        --- PASS: kuttl/harness/spark-pi-private-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev (92.62s)
        --- PASS: kuttl/harness/spark-history-server_openshift-true_spark-3.3.0-stackable0.0.0-dev_s3-use-tls-true (233.02s)
        --- PASS: kuttl/harness/spark-pi-private-s3_openshift-true_spark-3.3.0-stackable0.0.0-dev (95.15s)
        --- PASS: kuttl/harness/logging_openshift-true_spark-3.4.0-stackable0.0.0-dev_ny-tlc-report-0.1.0 (324.88s)
        --- PASS: kuttl/harness/spark-examples_openshift-true_spark-3.4.0-stackable0.0.0-dev (50.35s)
        --- PASS: kuttl/harness/logging_openshift-true_spark-3.3.0-stackable0.0.0-dev_ny-tlc-report-0.1.0 (334.88s)
        --- PASS: kuttl/harness/spark-examples_openshift-true_spark-3.3.0-stackable0.0.0-dev (47.70s)
        --- PASS: kuttl/harness/spark-history-server_openshift-true_spark-3.4.0-stackable0.0.0-dev_s3-use-tls-false (235.11s)
        --- PASS: kuttl/harness/spark-history-server_openshift-true_spark-3.4.0-stackable0.0.0-dev_s3-use-tls-true (264.63s)
        --- PASS: kuttl/harness/spark-pi-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev (93.35s)
        --- PASS: kuttl/harness/spark-ny-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev_s3-use-tls-false (135.54s)
        --- PASS: kuttl/harness/spark-pi-public-s3_openshift-true_spark-3.3.0-stackable0.0.0-dev (85.85s)
        --- PASS: kuttl/harness/spark-ny-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev_s3-use-tls-true (163.62s)
        --- PASS: kuttl/harness/spark-history-server_openshift-true_spark-3.3.0-stackable0.0.0-dev_s3-use-tls-false (261.10s)
        --- PASS: kuttl/harness/resources_openshift-true_spark-3.4.0-stackable0.0.0-dev (141.23s)
        --- PASS: kuttl/harness/spark-ny-public-s3_openshift-true_spark-3.3.0-stackable0.0.0-dev_s3-use-tls-true (174.72s)
        --- PASS: kuttl/harness/resources_openshift-true_spark-3.3.0-stackable0.0.0-dev (137.79s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3-image_openshift-true_spark-3.4.0-stackable0.0.0-dev_ny-tlc-report-0.1.0 (133.19s)
        --- PASS: kuttl/harness/spark-ny-public-s3_openshift-true_spark-3.3.0-stackable0.0.0-dev_s3-use-tls-false (144.36s)

adwk67 commented 1 year ago

bors merge

bors[bot] commented 1 year ago

Pull request successfully merged into main.

Build succeeded!

The publicly hosted instance of bors-ng is deprecated and will go away soon.

If you want to self-host your own instance, instructions are here. For more help, visit the forum.

If you want to switch to GitHub's built-in merge queue, visit their help page.

All tests passed