fix: switch osx build to codesign and notarytool

lidel commented 7 months ago

This PR replaces gon (no longer maintained tool for signing and notarizing macOS software on CI) with modern/new CLI tools provided by Apple: codesign and notarytool.

To be honest, I've been fixing signing and/or notarization of dist.ipfs.tech or ipfs-desktop at least once a year due to tool changes /deprecations.

Hopefully, using official tools from Apple will reduce surface for breakage.

Closes #1066

TODO

[x] remove unmaintained gon
[x] evaluate rcodesign
[x] signing with codesign
[x] notarization with notarytool
- [x] refactor .sh script to notarize final .zip (with signed binary inside) (to fix this error)
[x] concurrency protection on master branch
[x] document where we would plug rcodesign if we ever have to move away from macOS worker and Apple tools

Appendix: why we are not using `rcodesign` right now

This PR was exploring switching from macos runner to linux one that does signing with rcodesign mentioned in https://github.com/ipfs/distributions/issues/1066#issuecomment-2045165980 (Used by Mozila for signing and notarizing things like Tor Browser).

The downside was that, iiuc (not a macOS person), we would need to generate new secrets related to notarization, because old user/app-specific pass no longer work with rcodesign.

New things are: APPLE_APIKEY_ISSUER_ID, APPLE_APIKEY_ID, and APPLE_APIKEY_FILE, and they likely require uber-admin of Apple Developer Org to do special dance to generate them, which is doable, but would take multiple days or weeks to do, bunch of unknowns / extra work because of where we are with IPFS/PL/Shipyard nucleation state.

To avoid unnecessary time sink, I'm parking this for now.

CLI Tools codesign and notarytool from apple seem to be good enough these days. This is approach big projects like NodeJS use (https://github.com/ipfs/distributions/issues/1066#issuecomment-2045165980), and allows us to fix signing and notarization without having to switch org and generating new secrets.

Obligatory:

Hieronymus Bosch, The Process of fixing Apple Notarization, oil on wood, 1475.

github-actions[bot] commented 7 months ago

Diff of Changes

Old: /ipns/dist.ipfs.tech at /ipfs/QmRvzCoSNBufETvKpMWu8pZP4pGmLSdnECFyhT2CMVxQmx New: /ipfs/QmRD74Aod662LmA9MPUABgBEeMU44mGDjbeY9BnSapiL4w

diff --new-file -u --recursive a/go-ipfs/v0.28.0-rc1/build-info b/go-ipfs/v0.28.0-rc1/build-info
--- a/go-ipfs/v0.28.0-rc1/build-info    1970-01-01 00:00:00.000000000 +0000
+++ b/go-ipfs/v0.28.0-rc1/build-info    2024-04-10 13:43:23.782413772 +0000
@@ -0,0 +1,4 @@
+go version go1.22.2 linux/amd64
+git sha of code: a91640f8b6963adbb1c71376f88e6f97c3132af0
+Linux ip-10-0-103-254 6.2.0-1018-aws #18~22.04.1-Ubuntu SMP Wed Jan 10 22:54:16 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
+built on Wed Apr 10 13:33:58 UTC 2024
diff --new-file -u --recursive a/go-ipfs/v0.28.0-rc1/dist.json b/go-ipfs/v0.28.0-rc1/dist.json
--- a/go-ipfs/v0.28.0-rc1/dist.json 1970-01-01 00:00:00.000000000 +0000
+++ b/go-ipfs/v0.28.0-rc1/dist.json 2024-04-10 13:43:23.786413788 +0000
@@ -0,0 +1,26 @@
+{
+  "id": "go-ipfs",
+  "version": "v0.28.0-rc1",
+  "releaseLink": "go-ipfs/v0.28.0-rc1",
+  "name": "go-ipfs",
+  "owner": "ipfs",
+  "description": "kubo (go-ipfs) is the earliest and most widely used implementation of IPFS. It includes:\n- an IPFS daemon server\n- extensive command line tooling\n- an HTTP RPC API for controlling the node\n- an HTTP Gateway for serving content to HTTP browsers\n",
+  "date": "April 10, 2024",
+  "platforms": {
+    "darwin": {
+      "name": "darwin Binary",
+      "archs": {
+        "amd64": {
+          "link": "/go-ipfs_v0.28.0-rc1_darwin-amd64.tar.gz",
+          "cid": "QmUHefCeGCPk7MAwksUUceb8icHWqdmiDEEZBSGLFU2LuS",
+          "sha512": "5416a8d21456da96344dddc530812e047989b55117419003be6ebb41d550d654b87576ac43db54d4a6739af3d289b64d41d54f8991bd39f2aeb5d6a1065e31d5"
+        }
+      }
+    }
+  },
+  "source": {
+    "link": "/go-ipfs-source.tar.gz",
+    "cid": "QmZgF8dWPaH4o2MKho3DfSWZKttBntLm8CaRu1HiCKuzS8",
+    "sha512": "9a53d0937cc47b90317fcd28fd9d0d01fd7fe6165971f69b572421874d0a6636aad3725109b032131d14976d076054b4f1659ef21307bb84649ad30bf29bcdf6"
+  }
+}
Binary files a/go-ipfs/v0.28.0-rc1/go-ipfs-source.tar.gz and b/go-ipfs/v0.28.0-rc1/go-ipfs-source.tar.gz differ
diff --new-file -u --recursive a/go-ipfs/v0.28.0-rc1/go-ipfs-source.tar.gz.cid b/go-ipfs/v0.28.0-rc1/go-ipfs-source.tar.gz.cid
--- a/go-ipfs/v0.28.0-rc1/go-ipfs-source.tar.gz.cid 1970-01-01 00:00:00.000000000 +0000
+++ b/go-ipfs/v0.28.0-rc1/go-ipfs-source.tar.gz.cid 2024-04-10 13:43:23.850414041 +0000
@@ -0,0 +1 @@
+QmZgF8dWPaH4o2MKho3DfSWZKttBntLm8CaRu1HiCKuzS8
diff --new-file -u --recursive a/go-ipfs/v0.28.0-rc1/go-ipfs-source.tar.gz.sha512 b/go-ipfs/v0.28.0-rc1/go-ipfs-source.tar.gz.sha512
--- a/go-ipfs/v0.28.0-rc1/go-ipfs-source.tar.gz.sha512  1970-01-01 00:00:00.000000000 +0000
+++ b/go-ipfs/v0.28.0-rc1/go-ipfs-source.tar.gz.sha512  2024-04-10 13:43:23.850414041 +0000
@@ -0,0 +1 @@
+9a53d0937cc47b90317fcd28fd9d0d01fd7fe6165971f69b572421874d0a6636aad3725109b032131d14976d076054b4f1659ef21307bb84649ad30bf29bcdf6  go-ipfs-source.tar.gz
Binary files a/go-ipfs/v0.28.0-rc1/go-ipfs_v0.28.0-rc1_darwin-amd64.tar.gz and b/go-ipfs/v0.28.0-rc1/go-ipfs_v0.28.0-rc1_darwin-amd64.tar.gz differ
diff --new-file -u --recursive a/go-ipfs/v0.28.0-rc1/go-ipfs_v0.28.0-rc1_darwin-amd64.tar.gz.cid b/go-ipfs/v0.28.0-rc1/go-ipfs_v0.28.0-rc1_darwin-amd64.tar.gz.cid
--- a/go-ipfs/v0.28.0-rc1/go-ipfs_v0.28.0-rc1_darwin-amd64.tar.gz.cid   1970-01-01 00:00:00.000000000 +0000
+++ b/go-ipfs/v0.28.0-rc1/go-ipfs_v0.28.0-rc1_darwin-amd64.tar.gz.cid   2024-04-10 13:43:23.998414628 +0000
@@ -0,0 +1 @@
+QmUHefCeGCPk7MAwksUUceb8icHWqdmiDEEZBSGLFU2LuS
diff --new-file -u --recursive a/go-ipfs/v0.28.0-rc1/go-ipfs_v0.28.0-rc1_darwin-amd64.tar.gz.sha512 b/go-ipfs/v0.28.0-rc1/go-ipfs_v0.28.0-rc1_darwin-amd64.tar.gz.sha512
--- a/go-ipfs/v0.28.0-rc1/go-ipfs_v0.28.0-rc1_darwin-amd64.tar.gz.sha512    1970-01-01 00:00:00.000000000 +0000
+++ b/go-ipfs/v0.28.0-rc1/go-ipfs_v0.28.0-rc1_darwin-amd64.tar.gz.sha512    2024-04-10 13:43:23.998414628 +0000
@@ -0,0 +1 @@
+5416a8d21456da96344dddc530812e047989b55117419003be6ebb41d550d654b87576ac43db54d4a6739af3d289b64d41d54f8991bd39f2aeb5d6a1065e31d5  /Users/runner/work/distributions/distributions/releases/go-ipfs/v0.28.0-rc1/go-ipfs_v0.28.0-rc1_darwin-amd64.tar.gz
diff --new-file -u --recursive a/go-ipfs/v0.28.0-rc1/results b/go-ipfs/v0.28.0-rc1/results
--- a/go-ipfs/v0.28.0-rc1/results   1970-01-01 00:00:00.000000000 +0000
+++ b/go-ipfs/v0.28.0-rc1/results   2024-04-10 13:43:23.998414628 +0000
@@ -0,0 +1 @@
+, darwin, amd64, 
diff --new-file -u --recursive a/go-ipfs/versions b/go-ipfs/versions
--- a/go-ipfs/versions  2024-04-10 13:43:23.758413676 +0000
+++ b/go-ipfs/versions  2024-04-10 13:43:23.738413597 +0000
@@ -107,3 +107,4 @@
 v0.27.0-rc1
 v0.27.0-rc2
 v0.27.0
+v0.28.0-rc1
diff --new-file -u --recursive a/kubo/v0.28.0-rc1/build-info b/kubo/v0.28.0-rc1/build-info
--- a/kubo/v0.28.0-rc1/build-info   1970-01-01 00:00:00.000000000 +0000
+++ b/kubo/v0.28.0-rc1/build-info   2024-04-10 13:43:24.062414882 +0000
@@ -0,0 +1,4 @@
+go version go1.22.2 linux/amd64
+git sha of code: a91640f8b6963adbb1c71376f88e6f97c3132af0
+Linux ip-10-0-103-254 6.2.0-1018-aws #18~22.04.1-Ubuntu SMP Wed Jan 10 22:54:16 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
+built on Wed Apr 10 13:33:06 UTC 2024
diff --new-file -u --recursive a/kubo/v0.28.0-rc1/dist.json b/kubo/v0.28.0-rc1/dist.json
--- a/kubo/v0.28.0-rc1/dist.json    1970-01-01 00:00:00.000000000 +0000
+++ b/kubo/v0.28.0-rc1/dist.json    2024-04-10 13:43:24.062414882 +0000
@@ -0,0 +1,26 @@
+{
+  "id": "kubo",
+  "version": "v0.28.0-rc1",
+  "releaseLink": "kubo/v0.28.0-rc1",
+  "name": "kubo",
+  "owner": "ipfs",
+  "description": "kubo (go-ipfs) is the earliest and most widely used implementation of IPFS. It includes:\n- an IPFS daemon server\n- extensive command line tooling\n- an HTTP RPC API for controlling the node\n- an HTTP Gateway for serving content to HTTP browsers\n",
+  "date": "April 10, 2024",
+  "platforms": {
+    "darwin": {
+      "name": "darwin Binary",
+      "archs": {
+        "amd64": {
+          "link": "/kubo_v0.28.0-rc1_darwin-amd64.tar.gz",
+          "cid": "QmfB9o5ES8138Li7D2VkyYnx4ukgzCNSrtVvwsGXm8Cn33",
+          "sha512": "84aadc9cadc7ad0c2d14a9f14eb1f46c8b68663e8a2f9ce53042d6b543dc6bfdd13fc2d6b47884b447edda79ebac557c30a10ebbcfdb641742fd222596c43902"
+        }
+      }
+    }
+  },
+  "source": {
+    "link": "/kubo-source.tar.gz",
+    "cid": "QmZHL1BCRJsD8bj9MShsMPfBG8fZLX93LCSRADUKpJvQnd",
+    "sha512": "37ed9a730e4cfc1ece9f6cfa0745506b4b30dcbfadcf04a25ada69911876dab9f76188033abb5dd09d74f173cc2b54ada24fb666035ad8a61c55a47a9430b17d"
+  }
+}
Binary files a/kubo/v0.28.0-rc1/kubo-source.tar.gz and b/kubo/v0.28.0-rc1/kubo-source.tar.gz differ
diff --new-file -u --recursive a/kubo/v0.28.0-rc1/kubo-source.tar.gz.cid b/kubo/v0.28.0-rc1/kubo-source.tar.gz.cid
--- a/kubo/v0.28.0-rc1/kubo-source.tar.gz.cid   1970-01-01 00:00:00.000000000 +0000
+++ b/kubo/v0.28.0-rc1/kubo-source.tar.gz.cid   2024-04-10 13:43:24.122415120 +0000
@@ -0,0 +1 @@
+QmZHL1BCRJsD8bj9MShsMPfBG8fZLX93LCSRADUKpJvQnd
diff --new-file -u --recursive a/kubo/v0.28.0-rc1/kubo-source.tar.gz.sha512 b/kubo/v0.28.0-rc1/kubo-source.tar.gz.sha512
--- a/kubo/v0.28.0-rc1/kubo-source.tar.gz.sha512    1970-01-01 00:00:00.000000000 +0000
+++ b/kubo/v0.28.0-rc1/kubo-source.tar.gz.sha512    2024-04-10 13:43:24.122415120 +0000
@@ -0,0 +1 @@
+37ed9a730e4cfc1ece9f6cfa0745506b4b30dcbfadcf04a25ada69911876dab9f76188033abb5dd09d74f173cc2b54ada24fb666035ad8a61c55a47a9430b17d  kubo-source.tar.gz
Binary files a/kubo/v0.28.0-rc1/kubo_v0.28.0-rc1_darwin-amd64.tar.gz and b/kubo/v0.28.0-rc1/kubo_v0.28.0-rc1_darwin-amd64.tar.gz differ
diff --new-file -u --recursive a/kubo/v0.28.0-rc1/kubo_v0.28.0-rc1_darwin-amd64.tar.gz.cid b/kubo/v0.28.0-rc1/kubo_v0.28.0-rc1_darwin-amd64.tar.gz.cid
--- a/kubo/v0.28.0-rc1/kubo_v0.28.0-rc1_darwin-amd64.tar.gz.cid 1970-01-01 00:00:00.000000000 +0000
+++ b/kubo/v0.28.0-rc1/kubo_v0.28.0-rc1_darwin-amd64.tar.gz.cid 2024-04-10 13:43:24.258415659 +0000
@@ -0,0 +1 @@
+QmfB9o5ES8138Li7D2VkyYnx4ukgzCNSrtVvwsGXm8Cn33
diff --new-file -u --recursive a/kubo/v0.28.0-rc1/kubo_v0.28.0-rc1_darwin-amd64.tar.gz.sha512 b/kubo/v0.28.0-rc1/kubo_v0.28.0-rc1_darwin-amd64.tar.gz.sha512
--- a/kubo/v0.28.0-rc1/kubo_v0.28.0-rc1_darwin-amd64.tar.gz.sha512  1970-01-01 00:00:00.000000000 +0000
+++ b/kubo/v0.28.0-rc1/kubo_v0.28.0-rc1_darwin-amd64.tar.gz.sha512  2024-04-10 13:43:24.258415659 +0000
@@ -0,0 +1 @@
+84aadc9cadc7ad0c2d14a9f14eb1f46c8b68663e8a2f9ce53042d6b543dc6bfdd13fc2d6b47884b447edda79ebac557c30a10ebbcfdb641742fd222596c43902  /Users/runner/work/distributions/distributions/releases/kubo/v0.28.0-rc1/kubo_v0.28.0-rc1_darwin-amd64.tar.gz
diff --new-file -u --recursive a/kubo/v0.28.0-rc1/results b/kubo/v0.28.0-rc1/results
--- a/kubo/v0.28.0-rc1/results  1970-01-01 00:00:00.000000000 +0000
+++ b/kubo/v0.28.0-rc1/results  2024-04-10 13:43:24.258415659 +0000
@@ -0,0 +1 @@
+, darwin, amd64, 
diff --new-file -u --recursive a/kubo/versions b/kubo/versions
--- a/kubo/versions 2024-04-10 13:43:24.042414803 +0000
+++ b/kubo/versions 2024-04-10 13:43:24.022414723 +0000
@@ -37,3 +37,4 @@
 v0.27.0-rc1
 v0.27.0-rc2
 v0.27.0
+v0.28.0-rc1

lidel commented 7 months ago

Good news, confirmed both signing and notarization setup from https://github.com/ipfs/distributions/pull/1078/commits/69c061a8fd781ea0f2b9995dd574c6d176628629 produced amd64 Mach-O binary that is signed and notarized.

Tested on macMini with macOS 13.2.1 and ipfs --version was executed correctly, was not blocked by Apple's Gatekeeper system.

Now I will clean things up and merge back into master and then into #1077. This way it will be easier to git blame signing code.

github-actions[bot] commented 7 months ago

This change produced no new differences in built artifacts.

ipfs / distributions