search

mesosphere / spark-build

Used to build the mesosphere/spark docker image and the DC/OS Spark package
52 stars 34 forks source link

Modify cluster setup method to fix failing 'hdfs-kerberos-delete-terasort-files' job on soak114 #544

Closed alembiewski closed 5 years ago

alembiewski commented 5 years ago

What changes were proposed in this pull request?

Fixes failing job (https://soak114s.testing.mesosphe.re/#/jobs/detail/spark-hdfs-kerberos-terasort) on soak114 cluster.

Before the main job execution, the clean-up job hdfs-kerberos-delete-terasort-files gets created, but it fails to execute with no cluster attached error. After some investigation, it was decided to replace the old python script, used in soak110, with dcos cli command.

How were these changes tested?

Tested on majority of soak clusters with overriding $GIT_BRANCH job parameter.

Release Notes

n/a

alembiewski commented 5 years ago

@akirillov, yes, the script could be found in soak-cluster-configurations repository, and it looks like it was simply copy-pasted from one soak cluster folder to another without any modifications.