stackabletech / spark-k8s-operator

Operator for Apache Spark-on-Kubernetes for Stackable Data Platform
https://stackable.tech
Other
51 stars 2 forks source link

[Merged by Bors] - feat: Set explicit resources on all containers #249

Closed razvan closed 1 year ago

razvan commented 1 year ago

Description

Part of: https://github.com/stackabletech/issues/issues/394

Definition of Done Checklist

# Author
- [x] Changes are OpenShift compatible
- [x] CRD changes approved
- [x] Helm chart can be installed and deployed operator works
- [x] Integration tests passed (for non trivial changes)
# Reviewer
- [ ] Code contains useful comments
- [ ] (Integration-)Test cases added
- [ ] Documentation added or updated
- [ ] Changelog updated
- [ ] Cargo.toml only contains references to git tags (not specific commits or branches)
# Acceptance
- [x] Feature Tracker has been updated
- [x] Proper release label has been added

Once the review is done, comment bors r+ (or bors merge) to merge. Further information

razvan commented 1 year ago

Tests almost pass. Will retry the resources test:

--- FAIL: kuttl (1055.12s)
    --- FAIL: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/spark-examples_openshift-true_spark-3.4.0-stackable0.0.0-dev (82.57s)
        --- PASS: kuttl/harness/spark-ny-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev_s3-use-tls-true (186.31s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev (147.56s)
        --- PASS: kuttl/harness/logging_openshift-true_spark-3.4.0-stackable0.0.0-dev_ny-tlc-report-0.1.0 (444.98s)
        --- PASS: kuttl/harness/spark-history-server_openshift-true_spark-3.4.0-stackable0.0.0-dev_s3-use-tls-true (287.87s)
        --- PASS: kuttl/harness/spark-pi-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev (107.96s)
        --- PASS: kuttl/harness/spark-pi-private-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev (118.64s)
        --- FAIL: kuttl/harness/resources_openshift-true_spark-3.4.0-stackable0.0.0-dev (257.08s)
        --- PASS: kuttl/harness/smoke_openshift-true_spark-3.4.0-stackable0.0.0-dev_s3-use-tls-true (197.46s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3-image_openshift-true_spark-3.4.0-stackable0.0.0-dev_ny-tlc-report-0.1.0 (150.53s)
FAIL
razvan commented 1 year ago

Resources test now green too:

--- PASS: kuttl (107.98s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/resources_openshift-true_spark-3.4.0-stackable0.0.0-dev (96.07s)
PASS
razvan commented 1 year ago

All tests pass:

--- PASS: kuttl (949.01s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/spark-ny-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev_s3-use-tls-true (178.71s)
        --- PASS: kuttl/harness/smoke_openshift-true_spark-3.4.0-stackable0.0.0-dev_s3-use-tls-true (195.18s)
        --- PASS: kuttl/harness/logging_openshift-true_spark-3.4.0-stackable0.0.0-dev_ny-tlc-report-0.1.0 (446.77s)
        --- PASS: kuttl/harness/spark-examples_openshift-true_spark-3.4.0-stackable0.0.0-dev (80.38s)
        --- PASS: kuttl/harness/resources_openshift-true_spark-3.4.0-stackable0.0.0-dev (100.90s)
        --- PASS: kuttl/harness/spark-history-server_openshift-true_spark-3.4.0-stackable0.0.0-dev_s3-use-tls-true (298.60s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3-image_openshift-true_spark-3.4.0-stackable0.0.0-dev_ny-tlc-report-0.1.0 (147.50s)
        --- PASS: kuttl/harness/spark-pi-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev (115.56s)
        --- PASS: kuttl/harness/spark-pi-private-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev (113.23s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev (149.03s)
PASS
sbernauer commented 1 year ago

Please also update the docs (and maybe kuttl tests) to the new default values. BTW there are rust test failures

razvan commented 1 year ago
--- PASS: kuttl (967.88s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/spark-examples_openshift-true_spark-3.4.0-stackable0.0.0-dev (83.53s)
        --- PASS: kuttl/harness/smoke_openshift-true_spark-3.4.0-stackable0.0.0-dev_s3-use-tls-true (206.75s)
        --- PASS: kuttl/harness/spark-ny-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev_s3-use-tls-true (182.25s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev (143.34s)
        --- PASS: kuttl/harness/spark-history-server_openshift-true_spark-3.4.0-stackable0.0.0-dev_s3-use-tls-true (297.30s)
        --- PASS: kuttl/harness/resources_openshift-true_spark-3.4.0-stackable0.0.0-dev (100.99s)
        --- PASS: kuttl/harness/spark-pi-public-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev (120.84s)
        --- PASS: kuttl/harness/logging_openshift-true_spark-3.4.0-stackable0.0.0-dev_ny-tlc-report-0.1.0 (451.95s)
        --- PASS: kuttl/harness/spark-pi-private-s3_openshift-true_spark-3.4.0-stackable0.0.0-dev (120.39s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3-image_openshift-true_spark-3.4.0-stackable0.0.0-dev_ny-tlc-report-0.1.0 (153.78s)
PASS
razvan commented 1 year ago

bors merge

bors[bot] commented 1 year ago

Pull request successfully merged into main.

Build succeeded!

The publicly hosted instance of bors-ng is deprecated and will go away soon.

If you want to self-host your own instance, instructions are here. For more help, visit the forum.

If you want to switch to GitHub's built-in merge queue, visit their help page.