search

blueprint-freespeech / ricochet-build

Repo for building ricochet-refresh
https://github.com/blueprint-freespeech/ricochet-refresh
BSD 3-Clause "New" or "Revised" License
3 stars 4 forks source link

ricochet-refresh: export SOURCE_DATE_EPOCH #18

Closed NoisyCoil closed 6 months ago

NoisyCoil commented 6 months ago

This PR should be enough to make the linux-{i686,x86_64} deb package and tar.gz archive reproducible. The AppImages and bundles for other platforms are not. stable sha256sums:

2343c708d43bc11038cbc1cf6cbd2445cfcebe8efe91ecf0838da12de1832d50  ricochet-refresh-3.0.22-i386.appimage
005146aebc2a65ed45d6c71325a77b9806a6dd9c2b4c19bb2efe1b1d9dc124b7  ricochet-refresh-3.0.22-linux-i686.tar.gz
3bd29d896a9e2b05581614022bcfa9272618dc60990da01509d362f66bb9fb9e  ricochet-refresh-3.0.22-linux-x86_64.tar.gz
44d388c029c7f8729c438da6d22c00fc5c634618dbfa58e4f63ee1bddd941a51  ricochet-refresh-3.0.22-macos-aarch64.dmg
a1220cda56f243b190594913fede996151aac245a97e12bb002c8960cec6c7a1  ricochet-refresh-3.0.22-macos-x86_64.dmg
7a4962b67f52df8b9b018fecf54a35aa85b49b6cf8af4de306784395bc251ac7  ricochet-refresh-3.0.22-windows-i686-installer.exe
443dc1344de3d9961c532061abbe6744a49b009bda6fed8d0fb45312b9afb99f  ricochet-refresh-3.0.22-windows-i686.zip
c24cca0abf68449b0775156d9f1d48cc411d8e8e23635a1e55649762534ca7e0  ricochet-refresh-3.0.22-windows-x86_64-installer.exe
b84818e995af58a2cc0e8f19684ea2ff53e060f8389d41a8b225e66a4aee5a98  ricochet-refresh-3.0.22-windows-x86_64.zip
06fa7a0d67f8624345d81fe37d75a1ccdf4880a2ee68357801b467593652459e  ricochet-refresh-3.0.22-x86_64.appimage
4a4904fe69c2112c486e8d9f4a26c4b1a99f935436b212ac81b54c81d5d7b68f  ricochet-refresh_3.0.22_amd64.deb
4e33b665be152a000e523e5195a236246c1233de42968f0c94ead9faf61ee7c2  ricochet-refresh_3.0.22_i386.deb
NoisyCoil commented 6 months ago

Some explanation on the new commit. linuxdeploy internally uses mksquashfs with the -mkfs-time 0 option. As reported in https://github.com/AppImage/AppImageKit/issues/929, in theory this is intended to make builds reproducible (it sets some timestamps to zero, AIUI), but apparently it has the opposite effect: it neither makes the builds reproducible, nor it allows setting SOURCE_DATE_EPOCH to some specified value (mksquashfs -mkfs-time 0 fails if SOURCE_DATE_EPOCH is set) .

In order to obtain reproducible AppImages, Electrum is using the following workaround (see e.g. https://github.com/spesmilo/electrum/blob/9c94eb99f80fba89973978dd7f6d4727818ad7a3/contrib/build-linux/appimage/make_appimage.sh):

  1. they extract the AppImage they use to make AppImages (appimagetool in their case, linuxdeploy in ours)
  2. they replace the mksquashfs binary (which is embedded in the tool) with a wrapper script whose sole purpose is to remove the -mkfs-time 0 option from mksquashfs's command line and then call the real mksquashfs with the modified command line
  3. they execute the extracted and patched tool

By doing the above, one can freely set SOURCE_DATE_EPOCH, which is honored by mksquashfs and makes the AppImage artifact reproducible if the contents are the same.

The commit I just added to this PR does what Electrum are doing to linuxdeploy instead of appimagetool. I checked by deleting all artifacts (toolchain included) and then rebuilding them that all Linux artifacts are reproducible, with the following sha256sums:

ac48b9277789d3b826cf8c1fa9eb90174bc0e748d272e6eed2be30a5d68d4be1  ricochet-refresh-3.0.22-i386.appimage
005146aebc2a65ed45d6c71325a77b9806a6dd9c2b4c19bb2efe1b1d9dc124b7  ricochet-refresh-3.0.22-linux-i686.tar.gz
3bd29d896a9e2b05581614022bcfa9272618dc60990da01509d362f66bb9fb9e  ricochet-refresh-3.0.22-linux-x86_64.tar.gz
9e4f4e79cb09bcedfc68d35b469d5381511c4dae73abefb3a01031f281b35d6f  ricochet-refresh-3.0.22-macos-aarch64.dmg
ff42e832a5adec7bb5386fdbc1c344876486afd8555fda10997ae0638a88339b  ricochet-refresh-3.0.22-macos-x86_64.dmg
0d9d3299ae9488b1d7b89ab581b3d196104ed1fdb42d325755699cdc766f871b  ricochet-refresh-3.0.22-windows-i686-installer.exe
4e7610e1af54719c740b5f33aa8f975daf039db635231ac83e80df929d051bce  ricochet-refresh-3.0.22-windows-i686.zip
33d807adceb001aa9f63177ad17b591357273703604ac580243ecaf051d876c2  ricochet-refresh-3.0.22-windows-x86_64-installer.exe
cb5afe7b0d6a14c7d80205a4489b279689f4368a952cb178311e266761d4ba07  ricochet-refresh-3.0.22-windows-x86_64.zip
b78827e9c81ebb413936841ea6cde8085880de68f7bf653c64f7ec42c9887066  ricochet-refresh-3.0.22-x86_64.appimage
dccef17b70ba02e482120a75a8f8bb189bde95d3af217b4a9272c2600320970f  ricochet-refresh_3.0.22_amd64.deb
fc9710b3148f2ab2dbfebf735e252af90d58c06c5a463be60f8d5439cf87a663  ricochet-refresh_3.0.22_i386.deb

Note that some of the Linux shasums have changed with respect to the previous run because once SOURCE_DATE_EPOCH is set in projects/package (it wasn't up until the last commit), the tools in package honor it and change some timestamps. But Linux artifacts are reproducible, at least on my machine.

morganava commented 6 months ago

@NoisyCoil built your MR locally and got same hashes for all Linux targets! \o/

morganava commented 6 months ago

Ok looks mostly good to me, can you modify this so that the linuxdeploy updates (patching + removal of the timstamp arg) happen in a separate linuxdeploy project?

NoisyCoil commented 6 months ago

@NoisyCoil built your MR locally and got same hashes for all Linux targets! \o/

Great!

Ok looks mostly good to me, can you modify this so that the linuxdeploy updates (patching + removal of the timstamp arg) happen in a separate linuxdeploy project?

To make sure I understood, you want a separate linuxdeploy project just to patch linuxdeploy, and then import the patched version into package, right?

morganava commented 6 months ago

Yep exactly!

NoisyCoil commented 6 months ago

Sorry, pushed to the wrong remote. Not reverting this for the moment, I'll ping you when it's ready.

NoisyCoil commented 6 months ago

Done. Here are the sha256sums:

9d8aa4a0b9921c5ac3a8c1140601e3c34d2cd79a133190905c7e549edb55401f  out/package/package-3.0.23-linux-i686-e53778/ricochet-refresh-3.0.23-i386.appimage
cbbd59a071b85c563cb495913006bad81b55e30f5962f0657680d2277f3d5b89  out/package/package-3.0.23-linux-i686-e53778/ricochet-refresh-3.0.23-linux-i686.tar.gz
1fb25ea9d0037d82e9cd469b6b7c1ba193e9800af2d0a91247a9663703eed007  out/package/package-3.0.23-linux-i686-e53778/ricochet-refresh_3.0.23_i386.deb
6d575fd6b60f8fdf7b2abe747fa18696198aa95f2b221f0c7ba8dc4ef27d40dd  out/package/package-3.0.23-linux-x86_64-b83c1b/ricochet-refresh-3.0.23-linux-x86_64.tar.gz
0267c586cabcc4ecb642a51f9043fa9cdc2e9a3c3bff174a084871b017b2bbac  out/package/package-3.0.23-linux-x86_64-b83c1b/ricochet-refresh-3.0.23-x86_64.appimage
aa2b7189cfc5188261304934c5cfaacf827b6c38855f45fbdd9a556a48c57ed4  out/package/package-3.0.23-linux-x86_64-b83c1b/ricochet-refresh_3.0.23_amd64.deb

It turns out that the sed/dd hack is not necessary if linuxdeploy is unpacked first and then run.

morganava commented 6 months ago

Verified same hashes. Looks good to me!