Automate setup of new edge node

[bmpotter]

There are approximately 15 steps for a horizon user to do to an edge node, once the OS is installed, to get the node setup, registered with horizon, and verified. These steps can be automated to make the onboarding process quicker and less error prone. This is not intended for the production use cases rob has, but before most companies adopt a technology they take it for a test drive, which in our case could involve setting up a few edge nodes manually and trying one of the existing edge applications. This is the place the edge node setup automation could be useful.

But it is tricky to automate processes like this correctly, because covering up underlying steps can become problematic when something goes wrong, or the user needs more flexibility. So we must be intentional about minimizing these pitfalls.

David Martin wrote a good prototype of automating the horizon node setup, which we can use as a starting point, but it needs the following improvements:

• [ ] Restructure the code to make proper use of bash functions
• [ ] Provide Usage info
• [ ] Check all requirements up front, before doing anything: OS type/version, arch, input, env vars, etc.
• [ ] Be transparent, but succinct, about what the script is doing for the user, echoing certain commands where appropriate
• [ ] Enhance error handling to recognize even more problems
• [ ] Make the script idempotent, i.e. if an error occurs part way thru, once the user has fixed the problem they can simply run the script again and it will recognize what has already be done successfully.
• [ ] Make registering the pattern be optional (but the default)
• [ ] Have real command line parsing, so flags can be supported
• [ ] Remove loops looking for the firmware, images, and configstate values in hzn node list. A successful exit code from hzn register already assures these.

FYI, there are approx 15 steps that can be automated in https://console.test.cloud.ibm.com/docs/services/edge-fabric/adding-devices.html . Most of them can't be automated in the bluehorizon deb pkg because:

1) it's the steps for configuring apt and installing the deb pkgs, or 2) the steps require horizon user credentials (no way to give those to a deb pkg being installed), or 3) some steps are optional like starting the cpu2msghub example to verify the whole stack (no way to choose optional behavior in a deb install), or 4) installation of optional software (kafka is only required if using an edge service that sends data to kafka, so we wouldn't want bluehorizon to require always)

David Martin also wrote a script to create lots of devices: https://github.ibm.com/dcmartin/hzn-device-init and opened an issue: https://github.ibm.com/Edge-Fabric/project-mgmt/issues/131

Part of epic https://github.ibm.com/Edge-Fabric/project-summary/issues/35

[dcmartin]

I am changing the base script to only install Horizon. The biggest issue is the Kafka credentials; I now list the URL in the script to enable user to right click and open the IBM cloud page and copy credentials. It’s mostly idempotent now. I found that downloading the script and then running with bash was better than piping output. If we called the script setup-cpu2msghub then it would make sense to attempt the pattern, as well as capture the Kafka credentials. If we want it to be more generic, we should have a catalog of patterns, eg cpu2msghub, that can be selected; I did not know of any such catalog. I now have an operational home-assistant.io addon for the cpu2msghub pattern that collects the requisite pattern information separately from installation of horizon. It’s at https://github.com/dcmartin/hassio-addons in the cpu2msghub directory; this repo enables other HA users to select the cpu2msghub addon and install on their HA system.

[dcmartin]

It’s idempotent already

[dcmartin]

And automatic installation script in github.com/dcmartin/open-horizon/setup called init-devices.sh that scans LAN for vendor-specific devices (eg RaspberryPi) and then initiates the horizon installation and installs specified pattern; see template.json

[dcmartin]

I have re-implemented the initialization mechanism to utilize a service called hznsetup that listens for requests from devices and provides information required to register; a script run on the device consumes that JSON response and registers the device; it presumes that Horizon and it's pre-requisites have been installed; see https://github.ibm.com/dcmartin/open-horizon/blob/master/hznsetup/README.md

In addition, for the purposes of development and testing of the CI/CD process I have implemented a BASH script to register a node, including appropriate unregistration and when appropriate, complete purging and re-installation of the device's OH software; see https://github.ibm.com/dcmartin/open-horizon/blob/master/sh/nodereg.sh

To monitor the exchange and determine if either of the above two methods were successful in their use, I have implemented hznmonitor service as an Alpine Apache2 server; see https://github.ibm.com/dcmartin/open-horizon/blob/master/hznmonitor/README.md

To automate the installation of the OH device software, I have implemented the script aptget-horizon.sh (see https://github.ibm.com/dcmartin/open-horizon/blob/master/setup/aptget-horizon.sh) to install the bluehorizon package on DEBIAN LINUX and the OH horizon-container for macOS; it does not install pre-requisite software (e.g. Docker).

[dabooz]

This is superceded by the install script in anax/agent-install/agent-install.sh