[DCOS-58262] - Docker images in the manifest - Githubissues

mesosphere / spark-build

Used to build the mesosphere/spark docker image and the DC/OS Spark package

52 stars 34 forks source link

[DCOS-58262] - Docker images in the manifest #537

Closed rpalaznik closed 5 years ago

rpalaznik commented 5 years ago

What changes were proposed in this pull request?

Resolves DCOS-58262

List docker images for different spark distributions in manifest.json
The smoke test for custom docker images will now run automatically and use the images from manifest.json. The property for docker image is optional, but if any of the listed images does not exist, the test will be skipped.
The smoke test for custom docker images will use GroupBy example job instead of shuffle_app.py

How were these changes tested?

The test run in a standalone cluster in AWS.

Note: I listed docker images from my personal repository for now, to make CI build more meaningful. I'll remove them before merging.

rpalaznik commented 5 years ago

The tests were passed, so I've removed personal docker images from manifest.json. Proper ones should be added during the release procedure.