Adding archive checksums to releases

[petecooper]

Abstract To provide a verifiable method of confirming the downloaded Textpattern archive matches the official released archive by way of a shasum checksum. To check feasibility with Textpattern development team, and if accepted implement prior to 4.7.0 release.

Current status I've had a stab at putting this into effect with md5 checksums for each of the .zip and .tar.gz archives, but as this was implemented after the release of 4.6.2 it hasn't been battle tested in a release cycle. Oh, and MD5 has been proven to be insecure [1], which is bad.

Proposal Use shasum with 256-bit algorithm in place of md5, and create either of:

• a two-line digest of checksums for both .zip and .tar.gz (e.g. textpattern-4.7.0.SHA256SUM
• a corresponding text file with the checksum for each archive (e.g. textpattern-4.7.0.zip.SHA256SUM and textpattern-4.7.0.tar.gz.SHA256SUM).

The text file(s) should be made available as part of the release on GitHub, and transferred to the Textpattern site.

Benefits It's a verifiable way for users to confirm no funny business happens. Additionally, VirusTotal [2] uses a SHA256 hash for files, and if a single checksum tallies up between the official release, the downloaded file and the a clean report on VirusTotal, then that's an inexpensive sanity check for the archive's contents.

I've run textpattern-4.6.2.zip through the VirusTotal scanner [3] (clean bill of health, as you'd expect), and there's a visible reference to the file's checksum, which can be cross-checked:

Technicals shasum is included by default on many (?most) recent UNIX, GNU/Linux distributions, and macOS. As the text pattern-toolkit is a shell script, it's likely to be run on either UNIX, GNU/Linux or macOS platforms.

This commit [4] is a first attempt at adding shasum for the archives. Testing and feedback is warmly welcomed.

Workflow A minor change to the release workflow is needed -- checksum text file(s) should be distributed alongside the archive files. Some documentation for checking the archives once downloaded would be useful for completeness.

Feedback and advice

• Is this viable? It's a no-brainer to me, but I'm not involved in the release process.
• If so, is SHA256 an acceptable algorithm to use? Again, no-brainer to me, but etc.
• Do you want a digest or individual files?

That's all for now. Thanks for reading.

[1] https://en.wikipedia.org/wiki/MD5#Security [2] https://www.virustotal.com/ [3] https://www.virustotal.com/en/file/a868c05fc37108f2bb5e878cfbcdc61a82ce2646c4676cccb8105a6c6277be7a/ [4] https://github.com/petecooper/textpattern-toolbelt/commit/ea92221026d29927d08424a9a1a19418f8cf06a0

[philwareham]

@Bloke @vanmelick thoughts on this? Although I've seen checksum files for various things I've downloaded in the past I've never used them so I don't know anything about this.

Let me know how this would need to be displayed on the Textpattern.com website, if at all?

[Bloke]

I think a checksum is a worthy addition. It's easy to calculate on the archives and, yes, a link should be displayed alongside each download on the .com website if possible. We can add the files to the GitHub release too, as Pete says. And SHA256 seems like a good choice to me.

EDIT: Not sure whether to link to checksum files hosted on .com, or if it's safer to point to ones on public spaces in GitHub. My gut instinct is the latter. Openness, etc.

[philwareham]

OK, so just to get my head around it - the checksum is a totally separate file link that the user downloads (optionally) when they download the Zip (or gZip) - so I'd need to upload those files in the Textpattern.com CMS too - or link to a download from GitHub - yes?

[petecooper]

@philwareham -- correct, the checksum itself is separate from the archive, and could be a file, though it could also be a text string inline instead (custom field, perhaps?). The user can optionally refer to it to ensure that what they're downloading is what was released, though it shouldn't be a barrier to downloading.

This is how the Handbrake project handle it (inline SHA1 and SHA256): https://handbrake.fr/checksums.php

This is how the Transmission project handle it (inline SHA256 with link to VirusTotal): http://transmissionbt.com/download/

[philwareham]

I'd prefer the simple text string rather than another file, and it'd be relatively simple for me to roll that into the file download snippet on the forthcoming Textpattern.io site in a similar fashion to the two examples you gave.

If you can provide me the SHA256 for Textpattern release 4.6.2 I can see how to fit it into my templates (and GitHub).

[petecooper]

$ shasum -a 256 ./textpattern-4.6.2.zip
a868c05fc37108f2bb5e878cfbcdc61a82ce2646c4676cccb8105a6c6277be7a  ./textpattern-4.6.2.zip

$ shasum -a 256 ./textpattern-4.6.2.tar.gz 
77b12daf91a9a2762f9df7b410c43d05e7ab7a12e32614f534f49b910b3ec303  ./textpattern-4.6.2.tar.gz

[philwareham]

OK, I've built the SHA256 hashes into the new site (see grab). Not sure what the process is to generate these keys in the tool belt repo but maybe the note of you can do that bit.

Cheers!

[petecooper]

Looks great! This PR pulls in the code changes to make the checksums: https://github.com/textpattern/textpattern-toolbelt/pull/9 -- it generates 2x text files, and they can either be made available as part of the GitHub release (separate files) or used in the generation of the release info text if you'd prefer not to have them as separate files.

Edit: this issue can be closed if the code is merged.

[philwareham]

Brilliant - thanks! If you could possibly provide me those 2 text files generated against 4.6.2 I'll add those to the GitHub release, for completeness.

[petecooper]

Sure thing. Attached as a zip to preserve filenames.

textpattern-4.6.2-checksums.zip

[philwareham]

Cheers, let me know if this looks OK?: https://github.com/textpattern/textpattern/releases/tag/4.6.2

[petecooper]

Spot on. Thanks very much!