search

stackabletech / spark-k8s-operator

Operator for Apache Spark-on-Kubernetes for Stackable Data Platform
https://stackable.tech
Other
47 stars 2 forks source link

feat(tests): let spark distribute app jars over s3 #374

Closed razvan closed 3 months ago

razvan commented 3 months ago

:green_apple: CI: https://ci.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/spark-k8s-operator-it-custom/130/

Remove example jar files from this repo. Speed up tests by not having to copy app jars into the K8S cluster.

razvan commented 3 months ago

All tests pass:

--- PASS: kuttl (1035.76s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/spark-history-server_openshift-false_spark-3.5.1_s3-use-tls-true (255.40s)
        --- PASS: kuttl/harness/logging_openshift-false_spark-3.5.1_ny-tlc-report-0.1.0 (348.95s)
        --- PASS: kuttl/harness/pod_overrides_openshift-false_spark-3.5.1 (130.35s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3-image_openshift-false_spark-3.5.1_ny-tlc-report-0.1.0 (144.93s)
        --- PASS: kuttl/harness/spark-pi-public-s3_openshift-false_spark-3.5.1 (120.25s)
        --- PASS: kuttl/harness/smoke_openshift-false_spark-3.5.1_s3-use-tls-true (159.72s)
        --- PASS: kuttl/harness/delta-lake_spark-delta-lake-3.5.1_delta-3.1.0 (231.16s)
        --- PASS: kuttl/harness/resources_openshift-false_spark-3.5.1 (89.60s)
        --- PASS: kuttl/harness/iceberg_spark-3.5.1 (65.70s)
        --- PASS: kuttl/harness/spark-examples_openshift-false_spark-3.5.1 (54.55s)
        --- PASS: kuttl/harness/spark-ny-public-s3_openshift-false_spark-3.5.1_s3-use-tls-true (152.44s)
        --- PASS: kuttl/harness/spark-pi-private-s3_openshift-false_spark-3.5.1 (113.88s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3_openshift-false_spark-3.5.1 (158.12s)
PASS