search

Start9Labs / service-pipeline

Service packaging pipeline and coordination for StartOS
10 stars 0 forks source link

readme

Start9 Service Packaging for StartOS

Welcome! Thank you for your interest in contributing to the growing ecosystem of open software. We call the software applications that run on StartOS, "services". This distinction is made to differentiate from applications ("apps"), which are generally run on a client, and used to access server-side software ("services"). To run services on StartOS, a package of file components needs to be composed. This guide will dive into the basic structure of how to compose this package.

Check out the glossary to get acquainted with unfamiliar terms.

Let's get started!

Steps:

  1. Choose software to package
  2. Set up the development environment
  3. Try out the hello-world demo project
  4. Package the service
    1. Build a Dockerfile
    2. Create the file structure
    3. Format the package
  5. Test the service on StartOS
  6. Submit and/or distribute

Choosing Software

Almost any type of open source software can be run on StartOS. No matter what programming language, framework, database or interface the service has, it can be adapted for StartOS. This is made possible by the power of Docker containers (don't worry, we'll get to this). We do have a few recommendations for choosing a service that will perform optimally across platforms:

  1. It either has a web user interface (it can be interacted with in a web browser), or is server software that external applications or internal services can connect to. Please keep in mind that StartOS users are not expected to have SSH and/or CLI access.
    • The interfaces supported are: HTTP, TCP, and REST APIs
  2. It can be compiled for ARM (arm64v8) and/or (amd64)
  3. It can be served over TOR
  4. It creates a Docker image that is optimized for size (under 1GB) to save device space and expedite installation time

Development Environment

A basic development and testing environment includes:

  1. An Server One or Pro with latest StartOS
    • Use your own hardware to DIY
    • Purchase a device from the Start9 Store
    • x86/VM support coming soon
  2. A development machine
    • Linux is highly recommended, and this walkthrough will assume a Debian-based (Ubuntu) distribution

Dependencies - Recommended

These tools may or may not be necessary, depending on your environment and the package you are building.

Dependencies - Required

Demo with Hello World

Check your environment setup by building a demo project and installing it to StartOS.

  1. Get Hello World 
    git clone https://github.com/Start9Labs/hello-world-wrapper.git
cd hello-world-wrapper
  2. Build to create hello-world.s9pk 
    make
  3. Sideload & Run
    • In the StartOS web UI menu, navigate to Settings -> Sideload Service
    • Drag and drop or select the hello-world.s9pk from your filesystem to install
    • Once the service has installed, navigate to Services -> Hello World and click "Start"
    • Once the Health Check is successful, click "Launch UI" and verify you see the Hello World page

Package a service

The package file produced by this process has a s9pk extension. This file is what is installed to run a service on StartOS.

1. Build a Dockerfile

A Dockerfile defines the recipe for building the environment to run a service. Currently, StartOS only supports one Dockerfile per project (i.e. no Docker compose), so it should include any necessary database configurations. There are several methods to build a Dockerfile for your service.

First, check to see if the upstream project has already built one. This is usually your best source for finding Docker images that are compatible with ARM. Next, you can:

  1. Download an image from Docker Hub
  2. Make a new Dockerfile, and pull in an image the upstream project hosted on Docker Hub as the base
  3. Make a new Dockerfile, and pull in a small distribution base (eg. alpine) and compile the build environment yourself using the upstream project source code

After coding the build steps, build the Docker image using docker buildx, replacing the placeholder variables:

docker buildx build --tag start9/$(PKG_ID)/main:$(PKG_VERSION) --platform=linux/$(PLATFORM) -o type=docker,dest=image.tar .

The resulting docker-images/aarch64.tar or docker-images/x86_64.tar artifact (depending on if you used --platform=linux/arm64 or --platform=linux/amd64 is the Docker image that needs to be included in the s9pk package.

2. Create File Structure

Once we have a Docker image, we can create the service wrapper. A service wrapper is a repository of a new git committed project that "wraps" an existing project (i.e. the upstream project). It contains the set of metadata files needed to build a s9pk, define information displayed in the user interface, and establish the data structure of your package. This repository can exist on any hosted git server - it does not need to be a part of the Start9 GitHub ecosystem.

The following files should be included in the service wrapper repository:

3. Format package

Building the final s9pk artifact depends on the existence of the files listed above, and the execution of the following steps (which should be added to the Makefile):

start-sdk pack
# replace PKG_ID with your package identifier
start-sdk verify s9pk $(PKG_ID).s9pk

The verification step will provide details about missing files, or fields in the service manifest file.

That's it! You now have a package!

4. Testing

  1. Run the make command from the root folder of your wrapper repository to execute the build instructions defined in the MAKEFILE
  2. Install the package, via either:
    1. Drag and drop:
      • In the StartOS web UI menu, navigate to Settings -> Sideload Service
      • Drag and drop or select the <package>.s9pk from your filesystem to install
    2. Use the CLI:
      1. Create a config file with the IP address of the device running StartOS: 
        touch /etc/embassy/config.yaml
echo "host: <IP_ADDRESS_REPLACE_ME>" > /etc/embassy/config.yaml
# login with master password 
start-cli auth login
start-cli package install <PACKAGE_ID_REPLACE_ME>.s9pk

Installing...

  1. Once the service has installed, navigate to Services -> <Service Name> and click "Start"
  2. Check that the service operations function as expected by either launching the UI, or querying if a server application
  3. Check that each UI element on the service's page displays the proper information and is accurately formatted
  4. Ensure the service can be stopped, restarted, and upgraded (if applicable)

A service on eOS

5. Submission and Distribution

The s9pk file can be uploaded for distribution to any website, repository, or marketplace. You can also submit your package for publication consideration on a Start9 Marketplace by emailing us at submissions@start9labs.com or by contacting us in one of our community channels. Please include a link to the wrapper repository with a detailed README in the submission.

Advanced configuration

Scripting on StartOS

Start9 has developed a highly extensible scripting API for developers to create the best possible user experience. This is your toolkit for creating the most powerful service possible by enabling features such as:

Use is optional. To experiment, simply use the existing skeleton from the Hello World wrapper example, changing only the package version in the migration file.

Check out the specification here.

Support

Have a question? Need a hand? Please jump into our Community, or our Matrix Community Dev Channel.

FAQ

  1. I'm looking for a more guidance with this process, do you have any resources?
    • Yes, you can find our full service packaging specification (and more) in our developer docs.

Glossary

service - open software applications that run on StartOS

package - the composed set of a Docker image, a service manifest, and service instructions, icon, and license, that are formatted into a file with the s9pk extension using start-sdk

wrapper - the project repository that "wraps" the upstream project, and includes additionally necessary files for building and packaging a service for eOS

scripts - a set of developer APIs that enable advanced configuration options for services

start-sdk - the Software Development Toolkit used to package and verify services for StartOS

open source software - computer software that is released under a license in which the copyright holder grants users the rights to use, study, change, and distribute the software and its source code to anyone and for any purpose

upstream project - the original, source project code that is used as the base for a service

StartOS - a browser-based, graphical operating system for a personal server

s9pk - the file extension for the packaged service artifact needed to install and run a service on StartOS

back to top