chore: switch to signing bundled binaries

[dpc]

Re #4339

We would like to sign the Nix-bundled binaries that people actually can run on any machine, and not just the raw binaries that will work only on... well... the machine that built them.

[dpc]

Someone check if still reproduces.

[m1sterc001guy]

I get the same hash for all binaries underneath the v0.2.2-x86_64-linux/nixos folder, but I'm getting 404 errors for downloading 00762a03a3d862a2ca6272a21fdc50bda5d36c42.tar.gz. Is this expected?

[dpc]

I get the same hash for all binaries underneath the v0.2.2-x86_64-linux/nixos folder, but I'm getting 404 errors for downloading 00762a03a3d862a2ca6272a21fdc50bda5d36c42.tar.gz. Is this expected?

Possibly I had that reference cached, and it requires specified branch=master. Try now.

[elsirion]

gpg isn't part of our nix shell, should it be?

[elsirion]

Turns out bundling isn't reproducible. The culprit seems to be arx, which doesn't the bundle deterministically. I extracted two versions of the bundle and the extracted data appears to be identical:

while read p; do if ! diff fedimintd.d/$p fedimintd-remote.d/$p; then echo "diff in $p"; fi; done < all

Where all was a file generated with find . -type f from inside the respective extracted directory.

From diffing the extraction script it appears that the encoding algorithm differs between my two bundles:

$ diff <(head -n 90 fedimintd) <(head -n 90 fedimintd-remote)
5c5
< rm0=false ; rm1=false ; shared=true ; hash=2452087e64cd0bc1 ; tmpdir=/tmp
---
> rm0=false ; rm1=false ; shared=true ; hash=ca3bb2b7dfb3116 ; tmpdir=/tmp
88,90c88,90
< { tr '%' '\000' | sed 's|[+][+]|%|g ; s|[+]_|+|g' | head -c 4194304 ;} <<\+
< BZh91AY&SY�w1�[%�����������������������������������~�%������)�|��l�RБ���>*T�(%����}S�ԧ]�
PQ{}n++�Z���W���ԭ7�T�rv�v�=���<�:����x^΍�˺���)��pvӀi��n�ċ����qn�hs�%%=(%%%%%%%%%%%         ons���؛J�]�(
                                                                                  }�NXD%�+_�%%�%n���++�ݙ�;���{�sV��{j-j��vת��w��
< ��{wX���z��w�u]�S��n��twm�k������X    ��;z�ݘ㶶���]Y��{z����k׵��g�����ly��v��7���
                                                                                  q�r,w9�v�h%�(���x����:�v�|
---
> { tr '#' '\000' | sed 's|%%|#|g ; s|%_|%|g' | head -c 4194304 ;} <<\%
> BZh91AY&SY��Eb[#�����������������������������������~�#��}y����
����ܻ����i��pvӀh���qAE����*�a�z}�J�©��;n��}����;�J�{�

[dpc]

That hash= being different looks sus.

https://github.com/solidsnack/arx/blob/7561fed76bb613302d1ae104f0eb2ad13daa9fac/System/Posix/ARX/TMPXTools.hs#L52

Edit: Actually it's probably a hash over compressed data. Looking further ...

[dpc]

@elsirion Found it https://github.com/matthewbauer/nix-bundle/pull/103

[dpc]

gpg isn't part of our nix shell, should it be?

I don't think so. This is not going be widely used, and not only you need gpg but also you need to setup a key, publish it, etc.

[dpc]

Reproduces for me now between two different machines where it was different previously.

[justinmoon]

This is what I get on a mac from this branch:

$ just sign-release v0.2.2
./scripts/release/sign.sh v0.2.2
Building...
warning: not writing modified lock file of flake 'github:dpc/bundlers/e8aafe89a11ae0a5f3ce97d1d7d0fcfb354c79eb':
• Updated input 'nix-bundle':
    'github:dpc/nix-bundle/ec52fce86513d73e989c6cff0367956c6b07a1ba' (2024-02-21)
  → 'github:dpc/nix-bundle/8ab9acfe0a31805de7899eddb8f3312d4e34a13d' (2024-02-21)
error:
       … in the left operand of the update (//) operator

         at /nix/store/f5ykcnsjzkvri0nmcj64d221lkxhrzrf-source/flake.nix:52:19:

           51|                 inherit system;
           52|               })) // (if drv?program then { } else {
             |                   ^
           53|                 name =

       … <borked>

         at «none»:0: (source not available)

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: attribute 'aarch64-darwin' missing

       at /nix/store/18amc9dx7hgg3pqd70mzk9azaxap748i-source/flake.nix:12:20:

           11|       nix-bundle = { program, system }: let
           12|         nixpkgs' = nixpkgs.legacyPackages.${system};
             |                    ^
           13|         nix-bundle = import self { nixpkgs = nixpkgs'; };
error: Recipe `sign-release` failed on line 113 with exit code 1

[elsirion]

I'm not even sure if bundling is supposed to work on Mac and you'd get different hashes anyway. Would be cool to at least have the CLI tools published for mac, but not a blocker imo.

[dpc]

I'm not even sure if bundling is supposed to work on Mac and you'd get different hashes anyway.

I have added ${system} to paths, so everything is ready for publishing on other archs.

@justinmoon Seems like nix-bundle specifically not supports that architecture RN. https://github.com/matthewbauer/nix-bundle/blob/52c1817bbf76726c2e368eb95b754eaf44de4a53/flake.nix#L7

[justinmoon]

@dpc SGTM. Figured that's probably the case. 🚢