Build and Publishing Process

[hacdias]

There is an history of problems with Electron Builder, the package we're now using to build and deploy IPFS Desktop's binaries to GitHub Releases and also manage our auto-update mechanism.

Currently, there are two issues opened related to our build mechanism and auto-update:

1. ~The build for macOS is failing (#1244). A quick fix for this is just using the branch from where the fix comes from (https://github.com/electron-userland/electron-builder/pull/4414) while the fix is not released.~
2. There are user concerns about the update mechanism and the entities we reach to get the new binaries (#1189, #789). A possible fix for this is to distribute the binaries using our own server.

A suggestion that would remove the need for Electron Builder and, at the same time, allowing to keep with GitHub Releases (or not), is using Electron's own packager, which requires a bit more work, but it's the official way and workable. electron-packager repository.

For updates, we could use updater.electronjs.org with GitHub Releases to avoid the need for Electron Builder's auto-update too: https://electronjs.org/docs/tutorial/updates#updating-applications.

/cc @autonome @lidel

[lidel]

On fixing macOS build

Due to severity of the issue I'd say its ok to use branch version to fix this.

Note: use harcoded commit to ensure our build remains deterministic and future changes to that PR do not break our build:

-    "electron-builder": "^22.1.0",
+    "electron-builder": "https://github.com/electron-userland/electron-builder/tarball/db7cec39e9d370c750c6ac1a11cae4756f2d04fc/electron-builder.tar.gz",

On Replacing electron-builder with electron-packager / custom solution

I think if we plan to change existing setup, then we need to aim at self-hosted solution and full dogfooding from the start.

Doing it half-way does not seem to be worth the time, as the issue at hand (1) can be fixed with the above.

I see some electron-packager servers for self-hosting are pretty simple, eg. Nuts. Looks like its mostly path conventions.

I am thinking in the lines of:

• removal of Github Releases as the source of truth for installs and updates (it should not have binaries, only a link to a directory on IPFS)
• use of our Gateways for binary distribution and autoupdates
• creating CI setup to build a static install/autoupdate website with updated unsigned binary artifacts (build of release branch should also sign binaries and update dnslink for desktop.dist.ipfs.io)

The hard requirement here is to dogfood as much IPFS we can. If we use third party CDN it sends signal something is wrong with IPFS.

With the above we would not only remove awkward third parties from the picture, but create a template for other Electron apps to follow (could be a blogpost or a separate repo with scripts and instructions).

[hacdias]

On fixing macOS build

~When trying to replace the electron-builder package by the fixed commit, I can't npm install.~ Okay, this won't be as easy as we could imagine.

Electron Builder uses a build process that outputs the built package to out and beyond that the repository also contains multiple packages, being electron-builder located under packages/electron-builder.

Two solutions:

1. Publish a fork under `@hacdias/electron-builder``
2. Revert to Electron 6.x and Electron Builder 21.x.

On Replacing electron-builder with electron-packager

It is a bit more complicated than what it seems. electron-packager does what it says: it packages the app. It does not create installable binaries. I would suggest keeping using electron-builder with our own update server though.

[hacdias]

Update: macOS fix was merged. Waiting for release.

[SgtPooki]

@hacdias @lidel Do you have recommendations for where to take this now? For going the web3 native route, we could probably publish on valist and then handle auto-updating from there.

[lidel]

If we want to change anything, then my recommendation would be to harden supply chain by doing content addressing end-to-end:

• generating CIDs for our binaries and pinning these CIDs on 2+ services (e.g. Estuary and web3.storage)
• make autoupdate work without reliance on DNS and GitHub, in a way that is cryptographically verifiable

The "native way" for IPFS Desktop would be to do updates over IPFS (+IPNS), and not a third-party blockchain marketplace.

Complexity that comes with Valist

Valist was proposed in https://github.com/ipfs/ipfs-desktop/pull/2269 and iirc we did not follow up as it was not a good match for our needs. All it does from our point of view is delegated publishing and distribution via their own blockchain-based app store.

It does not provide any drop-in JS SDK solution for standalone autoupdate of Electron app, and does not allow for passing precomputed CID for publishing. This means it is not trustless: we need to trust Valist supply chain is not malicious, because they take our binaries and do the chunking on their side to produce CID.

All this means we don't improve much, really:

• If we need to write our own autoupdate code, and if we need to babysit some Ethereum private key, and we don't get any security benefit from content-addressing
• .. how is Valist better than the current setup
• .. or how it is less work than creating fully IPFS-native solution: writing autoupdate that polls IPNS, babysitting IPNS key and generating CID ourselves and pinning it using existing pinning services? IPIP351 will allow polling gateways for IPNS updates, no need for blockchain clients, just a simple HTTP GETs to a few gateways, and using the latest quorum one.

[SgtPooki]

Thanks @lidel, that makes sense and I like the idea of going the native IPFS route, dogfooding and such.

@bajtos, has your team done anything with regard to hardening updates for filecoin station as mentioned in https://github.com/ipfs/ipfs-desktop/issues/789, or do you have any plans for such a thing?

[bajtos]

Thanks for pinging me, @SgtPooki! The Station is using electron-builder and electron-updater right now, and we don't have plans (or bandwidth) to look into different approaches. However, if you find a reasonably easy way how to do content-addressable updates, then we are happy to follow your lead. Eventually 🙃 /cc @juliangruber