Stellio context broker

Stellio is an NGSI-LD compliant context broker developed by EGM. NGSI-LD is an Open API and Datamodel specification for context management published by ETSI).

Stellio is a FIWARE Generic Enabler. Therefore, it can be integrated as part of any platform “Powered by FIWARE”. FIWARE is a curated framework of open source platform components which can be assembled together with other third-party platform components to accelerate the development of Smart Solutions. For more information check the FIWARE Catalogue entry for Core Context. The roadmap of this FIWARE GE is described here.

You can find more info at the FIWARE developers website and the FIWARE website. The complete list of FIWARE GEs and Incubated FIWARE GEs can be found in the FIWARE Catalogue.

NGSI-LD Context Broker Feature Comparison

The NGSI-LD Specification is regularly updated and published by ETSI. The latest specification is version 1.8.1 which was published in April 2024.

• An Excel file detailing the current compatibility of the development version of the Stellio Context Broker against the features of the 1.8.1 specification can be downloaded here

Overview

Stellio is composed of 2 business services:

• Search service is in charge of managing the information context and handling the temporal (and geospatial) queries, it is backed by a TimescaleDB database
• Subscription service is in charge of managing subscriptions and subsequent notifications, it is backed by a TimescaleDB database

It is completed with:

• An API Gateway module that dispatches requests to downstream services
• A Kafka streaming engine that decouples communication inside the broker (and allows plugging other services seamlessly)

The services are based on the Spring Boot framework, developed in Kotlin, and built with Gradle.

Quick start

A quick way to start using Stellio is to use the provided docker-compose.yml file in the root directory (feel free to change the default passwords defined in the .env file):

docker-compose up -d && docker-compose logs -f

It will start all the services composing the Stellio context broker platform and expose them on the following ports:

• API Gateway: 8080
• Search service: 8083
• Subscription service: 8084

Please note that the environment and scripts are validated on Ubuntu and macOS. Some errors may occur on other platforms.

We also provide a configuration to deploy Stellio in a k8s cluster. For more information, please look in the stellio-k8s project

TROUBLESHOOTING

If Kafka container crash when starting, you may need to change "config/kafka/update_run.sh" line separator from CRLF to LF

Docker images tagging

Starting from version 2.0.0, a new scheme is used for tagging of Docker images:

• Releases are tagged with the version number, e.g., stellio/stellio-search-service:2.0.0
• latest tag is no longer used for releases as it can be dangerous (for instance, triggering an unwanted major upgrade)
• On each commit on the develop branch, an image with the latest-dev tag is produced, e.g., stellio/stellio-search-service:latest-dev

The version number is obtained during the build process by using the version information in the build.gradle.kts file.

Development

Developing on a service

Requirements:

• Java 21 (we recommend using sdkman! to install and manage versions of the JDK)

To develop on a specific service, you can use the provided docker-compose.yml file inside each service's directory, for instance:

cd search-service
docker-compose up -d && docker-compose logs -f

Then, from the root directory, launch the service:

./gradlew search-service:bootRun

Running the tests

Each service has a suite of unit and integration tests. You can run them without manually launching any external component, thanks to Spring Boot embedded test support and to the great TestContainers library.

For instance, you can launch the test suite for entity service with the following command:

./gradlew search-service:test

Building the project

To build all the services, you can just launch:

./gradlew build

It will compile the source code, check the code quality (thanks to detekt) and run the test suite for all the services.

For each service, a self executable jar is produced in the build/libs directory of the service.

If you want to build only one of the services, you can launch:

./gradlew search-service:build

Committing

Commits follow the Conventional Commits specification.

Code quality

Code formatting and standard code quality checks are performed by Detekt.

Detekt checks are automatically performed as part of the build and fail the build if any error is encountered.

• You may consider using a plugin like Save Actions that applies changed code refactoring and optimized imports on a save.

• You can enable Detekt support with the Detekt plugin.

• You can also setup a precommit hook to run detekt autocorrect automatically

Pre-commit

Automatic setup with pre-commit tool

(if you don't have Python installed, use the manual setup below)

• install pip install pre-commit
• then run pre-commit install

  Manual setup

• copy the script in config/detekt/detekt_auto_correct.sh in your .git/pre-commit file

Working locally with Docker images

To work locally with a Docker image of a service without publishing it to Docker Hub, you can follow the below instructions:

• Build a tar image:

./gradlew search-service:jibBuildTar

• Load the tar image into Docker:

docker load --input search-service/build/jib-image.tar

• Run the image:

docker run stellio/stellio-search-service:latest

Releasing a new version

• Merge develop into master

git checkout master
git merge develop

• Update version number in build.gradle.kts (allprojects.version near the bottom of the file) and in .env (STELLIO_DOCKER_TAG environment variable)
• Commit the modification using the following template message

git commit -am "chore: upgrade version to x.y.z"

• Push the modifications

git push origin master

The CI will then create and publish Docker images tagged with the published version number in https://hub.docker.com/u/stellio.

• On GitHub, check and publish the release notes in https://github.com/stellio-hub/stellio-context-broker/releases

Usage

To start using Stellio, you can follow the API quick start.

Minimal hardware requirements needed to run Stellio

The recommended system requirements may vary depending on factors such as the scale of deployment, usage patterns, and specific use cases. That said, here are the general guidelines for the minimum computer requirements:

• Processor: Dual-core processor or higher
• RAM: 4GB or higher (1.8GB is needed to just run it)
• Storage: At least 4GB of free disk space (3.8GB is needed to just run it)
• Operating System: Linux (recommended), macOS (also recommended), or Windows

Please note that these requirements may vary based on factors such as the size of your dataset, the number of concurrent users, and the overall complexity of your use case.

Further resources

For more detailed explanations on NGSI-LD or FIWARE:

• NGSI-LD v1.8.1 Specification
• NGSI-LD Primer
• NGSI-LD Tutorials
• FIWARE Training

License

Stellio is licensed under APL-2.0.

It mainly makes use of the following libraries and frameworks (dependencies of dependencies have been omitted):

© 2020 - 2024 EGM