Add Windows-focused compiling tooling to create ready-to-use binary for hopr-chat

[0xjjpa]

Introduction

As described in our docs, hopr-chat can be run inside a Windows computer using Docker. However, the requirements for having Docker installed inside an average Windows machine are not only high, but also cumbersome for a non-technical user (e.g. enabling Hyper-V, ensuring the machine has virtualisation capabilities, etc). To showcase hopr-chat and push for mass adoption, we are looking to ship hopr-chat as a standalone binary that can be quickly executed to connect to other running hopr-chat nodes.

Task Description

• [ ] Create a new task inside chat/package.json called build:binary:windows which runs the required commands to deliver a binary inside dist/windows/winx64-12.9.1. The architecture targeted is x64 and the node version to be used is 12.x (ideally 12.9.1, as 12.9.0 has some memory leak issues).
• [ ] Create a release able to be run inside Windows 10 Home and Windows 10 Pro editions, both on 64-bit architectures, able to be run as standalone binaries executable via “double-click”. In case some external dependencies need to be bundled alongside (e.g. wrtc.node), please provide the binary as a zip compressed folder which can be extracted and run alongside the dependencies.

Definition of Complete

The task is considered completed when a build workflow for the Windows Binary has been added to the project and a successful release can be used on any Windows 64-bit machine on a double-click fashion. Furthermore, the binary is able to execute through all the steps shown in our Documentation, specifically from Step 3 to 5. The easiest way to do so, is by running the binary multiple times inside the same computer with different ports and effectively sending messages across each other.

Testing Criteria

• [ ] The updated package.json includes all the required devDependencies to bundle, compile and create a distributable binary usable in any Windows 64-bit computer.
• [ ] The updated package.json includes the script build:binary:windows which will trigger a local build against the existing codebase to produce an executable binary.
• [ ] The executable binary can be packed and distributed as a single file in an .exe format or similar. If additional external dependencies need to be distributed alongside, then the distributable will be a .zip file which will be easily extracted and then run in any working directory.
• [ ] The binary will be able to be run in any Windows 10 * distribution (e.g. Home, Pro) and all needed dependencies will be included, such as a node runtime that can process hopr-chat logic.
• [ ] Upon executing the binary, a terminal screen as shown in our Documentation will be shown. The binary can also be run from Powershell, and opened via Visual Studio 2019, run and log-traced there using F5.
• [ ] The binary will be able to trigger an Administrator Firewall Access Request prompt upon usage, or automatically connect to a bootstrap node when Run as an Administrator is used.
• [ ] The binary can read .env file inside the same working directory to read env variables such as BOOTSTRAP_SERVER
• [ ] The binary can be used to successfully complete all steps (particularly 3 to 5) from our Documentation. More importantly, the binary can be used to effectively showcase the messaging workflow as seen in our docs.
• [ ] The binary is under 1 GB in disk space.

Notes

Some prior work was conducted (see below under “Prior Work”). For the deliverable, we are not only looking for the compiled binary, but also the required steps to build one, alongside the tooling or pipeline setup needed to enable so. If third party tooling is used (e.g. Azure Pipelines), please describe as such.

Prior Work

There's some previous work done on this task, which can be seen in https://github.com/hoprnet/hopr-core/issues/26 and https://github.com/hoprnet/hopr-core/issues/88 more recently. Right now, all effort on this direction can be seen in https://github.com/hoprnet/hopr-core/issues/86, and might need tasks like https://github.com/hoprnet/hopr-testing/issues/1 to be completed first as to ease the process.

Nexe

Multiple efforts were done using nexe, but we stumbled upon multiple issues. To mention just some:

• Dependencies were not bundled - https://github.com/nexe/nexe/issues/642
• Native bindings are not available - https://github.com/nexe/nexe/issues/776
• Node_modules are not included - https://github.com/nexe/nexe/issues/727 (similar to 642)
• Dependencies are not resolved - https://github.com/nexe/nexe/issues/717
• Some modules are not found - https://github.com/nexe/nexe/issues/745 (rollup efforts were done w/o success).
• Native options are undocumented - https://github.com/nexe/nexe/issues/335.

In short, some work was done here but even by bundling the entire node_modules folder would create a 1.2 GB binary and would yet still not work. In some cases, the nexe created binary would work, but as soon as the binary would be moved to another computer, it will fail to find the dependencies, and even shipping with a node_modules folder alongside would produce some errors such as the following:

Pkg

Another tool called pkg was also explored, and even with the included externals it would not work.

In this particular case, we were faced with an error KeccakState, which we had been unable to resolve and are trying to pin to ganache-core as seen in https://github.com/hoprnet/hopr-core-ethereum/issues/63.

So far that has stopped our progress using pkg so far, as that doesn't seem to be an issue with pkg itself but with the program so far. It's worth mentioning that running the program with node running locally in a Windows computer works without any issues, and that pkg also has similar issues as nexe handling external/native dependencies (see https://github.com/vercel/pkg/issues/663, https://github.com/vercel/pkg/issues/749, and some efforts in https://github.com/vercel/pkg/pull/837)

State of Art

Some projects are looked upon in order to use them as a reference as to mimic their successful setup. In particular, we had been taking a close look at IPFS as they have effectively being able to run an ipfs node based on JavaScript alongside a Desktop App (see https://github.com/ipfs-shipyard/ipfs-desktop).

We have noticed that IPFS created its own bundler, Aegir as to bundle some of their code as seen in their javascript implementation of ipfs, and are now actively looking to implement it. It sadly doesn't have TypeScript support yet, but there's an existing PR driving some of that conversation (i.e. does not provide TypeScript support, only jsdocs).

(It's worth mentioning some efforts had already been done with Aegir, but we now stumble with some issues from our own dependencies (i.e. ganache-core) that need to be handled, see screenshot below).

Some of the work in terms of distributed deliverables for libp2p based projects for IPFS can be seen in their root GUI repo - https://github.com/ipfs/ipfs-gui. From compiling ipfs into a single executable file (js-ipfs) to bundling into their desktop app can be found from there. For instance, we have yet to explore electron-builder which will be the next step to follow.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

This issue now has a funding of 80.0 DAI (80.0 USD @ $1.0/DAI) attached to it.

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
  • Want to chip in? Add your own contribution here.
  • Questions? Checkout Gitcoin Help or the Gitcoin Chat
  • $128,052.22 more funded OSS Work available on the Gitcoin Issue Explorer

[0xjjpa]

This has been completed in https://github.com/hoprnet/hopr-core/pull/106. We are still not embedding everything needed for a full “binary” per se, but right now it's enough for us to get started with node as a dependency for the time being. Closing.

[KiChjang]

@SCBuergel Do you mind cancelling the corresponding Gitcoin bounty for this issue?