Spark releases are built against multiple versions of Hadoop. Keeping track of what build to use can be irritating. Flintrock will now use the selected Hadoop version to automatically select the appropriate build of Spark to use.
This simplifies things for the user but changes the format of the two download-source options. Flintrock should somehow smooth the transition of formats for users.
TODO:
[x] Add changelog.
[x] Bump major version.
[x] Add test for new Hadoop build version utility.
[x] Explain how file name is generated, with reference to dist.apache.org.
[x] Detect old download source format and raise deprecation warning.
[ ] ~Print helper message on appropriate version of hadoop-aws to use. (?)~
Spark releases are built against multiple versions of Hadoop. Keeping track of what build to use can be irritating. Flintrock will now use the selected Hadoop version to automatically select the appropriate build of Spark to use.
This simplifies things for the user but changes the format of the two
download-sourceoptions. Flintrock should somehow smooth the transition of formats for users.TODO:
Related: #322