Closed elmiko closed 5 years ago
@tmckayus @crobby @Jiri-Kremser ptal
~one more thing i found while reading through the change-yaml.sh
file, i need to update the md5 based logic in there to use the new sha512 stuff.~
~scratch that, i was wrong~
scratch that, it's complicated.
from the archives, some time around may 2018 it looks like the spark team starting distributing only pgp signatures (.asc
files) and sha512 sums (.sha512
files). because there were a few backports that were released after this policy went into effect you see strange things like; spark version 2.3.0 carries md5 sum files but 2.2.3 does not.
the change-yaml.sh
script will attempt to download the md5 files to confirm the validity of the archive file and then use that md5 in the image.yaml
cekit file. this will cause the script file to exit with an error.
it would be easy if we could just switch to use the upstream sha512 sums to validate the archive and inform cekit about, unfortunately cekit only has support for md5, sha1, and sha256 from the schema file.
i think the best thing to do is use the sha512's to validate the archive, then calculate an md5 from the archive to put in the schema file. big downside here is that you will need to download the archive to calculate the md5.
another option is to remove the checksum altogether, but this seems like a sacrifice of better security practice for a savings in time during configuration.
fwiw, i made a request to the cekit project for a feature =)
I think while we're waiting for the cekit feature, we can just do the extra download.
ok, cool. i'll get that patch up
TODO
comment in f2f00bbe6156cfc8b33689d37bbf3ce2f094e4f6 in #80. cc @elmiko.TODO
comment in f2f00bbe6156cfc8b33689d37bbf3ce2f094e4f6 in #80. cc @elmiko.TODO
comment in f2f00bbe6156cfc8b33689d37bbf3ce2f094e4f6 in #80. cc @elmiko.added the sha512 changes to change-yaml.sh
, just need to update the docs and we should be good to go.
@tmckayus added more content about the script files, let me know what you think.
\o/
This change brings in support for Spark 2.4.0, it also updates the versioning on the incomplete images.