Set up build scripts for all ARM build artifacts and the firmware

[dl1com]

• [x] Add a folder build_scripts where buildscripts for the various artifacts reside. This enables re-usage of the scripts for the end-user as well as for the automated Github actions.

• [x] Add script create_sysroot.sh This script pulls the plutosdr-fw repo with a given Tag or Commit hash from the Analog Devices Github account and alters its content according to our needs (compilation of fftw, libconfig, libfec and liquid-dsp). The result is called custom sysroot. The sysroot gets zipped and uploaded to Github releases to be available end-users and the build_binaries.sh script (see below).

• [x] Add script build_standalone_binaries.sh This script also makes use of the custom sysroot artifact. As it would be to much effort to re-generate the custom sysroot on every build of the binaries (especially for Github actions), the chosen version of the custom sysroot artifact is downloaded from Github releases instead.

• [x] Add script build_firmware.sh This script makes use of the custom sysroot which was created with the script mentioned above. Additionally, it patches and configures the Linux Kernel according to our needs and configures the Linux userspace. Finally, it builds the firmware image called pluto.frm.

• [ ] Update manual.hnap.de accordingly

• No actions for the x86 binaries yet (they are already implemented as Github actions in .github/workflows)

[lukasostendorf]

I've been working on a first version of this. Current state:

create_sysroot.sh

• ARGS: needs path to the sysroot folder
• downloads and installs all additional libraries and header files to the sysroot folder.
• creates a .tar.gz of the finished sysroot folder in the directory where the script is located (build_scripts)

build_firmware.sh

• ARGS: needs path to Xilinx Toolchain installation; path to sysroot folder.
• if no plutosdr-fw folder is downloaded yet:
  • downloads the plutosdr-fw repo to build_scripts/plutosdr-fw
  • verifies that plutosdr-fw git tag matches the tag against which we want to build (v0.32)
  • applies RT kernel patch and makes changes to kernel and buildroot config
• call create_sysroot.sh
• build the standalone applications
• build the firmware
• copy the firmware image to the `build_scripts' folder

Open questions

I do not want to download the whole firmware repo from scratch every time. It takes much time to download and even more time to compile the linux kernel from scratch every time. What is a good way to check whether it is already downloaded? Currently plutosdr-fw download path is fixed and a check whether the dir exists is performed.

Do we want to let the user choose where to create the plutosdr-fw dir?

Kernel patching and changing the buildroot+kernel configuration must be only done once or it will fail. How to check if this has already been done? The current script does not properly handle this.

[dl1com]

Please see my comments in your commit https://github.com/HAMNET-Access-Protocol/HNAP4PlutoSDR/commit/05bd6211b5ce5f05ca07e61bf4cb6f6ed419b974

Open questions

I do not want to download the whole firmware repo from scratch every time. It takes much time to download and even more time to compile the linux kernel from scratch every time. What is a good way to check whether it is already downloaded? Currently plutosdr-fw download path is fixed and a check whether the dir exists is performed.

Do we want to let the user choose where to create the plutosdr-fw dir?

I'd propose a fixed directory inside a designated build directory within the repo.

Kernel patching and changing the buildroot+kernel configuration must be only done once or it will fail. How to check if this has already been done? The current script does not properly handle this.

I'd propose touching a flag file and check if it exists prior to applying the Kernel modifications. This method won't be sufficient for many cases (e.g. changing to a different Tag / Branch in plutosdr-fw), but for our current quasi-static use case (always v0.32), this would be working, I guess.

[dl1com]

While setting up CI for the build_standalone_binaries.sh script (https://github.com/HAMNET-Access-Protocol/HNAP4PlutoSDR/compare/develop...workflow), I became aware of something we might want to optimize. Currently, the custom sysroot would be built each time we trigger a build, which increases build time unnecessarily to about 3 minutes. We could reduce it by either caching the custom sysroot folder or downloading an already customized sysroot folder:

• Caching the custom sysroot folder so it persists between builds
  • Would be pretty easy to implement, but we would need something which indicates the workflow when to clean the cached sysroot folder so it gets rebuilt when necessary
  • Example for caching: https://docs.github.com/en/actions/configuring-and-managing-workflows/caching-dependencies-to-speed-up-workflows#example-using-the-cache-action
  • @lukasostendorf which indicator could be used to check if we have to rebuild the sysroot folder? If we find a suitable indicator, this would be my preferred solution.
• Downloading an already customized folder
  • This would be pretty similar to what was already intended for v1.0.0: we supply a tar with the custom sysroot e.g. in github releases and fetch it before running the build script for the binaries.
  • Though, it would be something like a circular dependency, as we are relying on something which has not been released yet. We would have to assign a proper tag/release name to the custom sysroot which is currently used for development first.
  • If we wanted to supply a custom sysroot to the users anyway, this would be a good solution. But as we have really good build scripts now, the users can easily build it on their own and I don't think it's necessary to supply it because of the users.