Cumulocity-IoT / cumulocity-analytics-management

Extends the standard cumulocity administration with dialog to add analytics builder extensions
Apache License 2.0
7 stars 3 forks source link
apama apama-analytics-builder cumulocity-extension cumulocity-iot cumulocity-package cumulocity-webapp extension iot-analytics streaming-analytics webapp

cumulocity-analytics-management

Content

Overview

This solution extends the functionality of the standard Cumulocity Streaming Analytics application by incorporating a plugin to manage and add Analytics Builder extensions. The current standard user interface lacks the capability to upload custom blocks packaged as .zip files. This plugin enhances the Streaming-Analytics UI by introducing these capabilities.

The extension allows you to manage the complete lifecycle of an extension:

Analytics Builder blocks are build using the Analytics Builder Block SDK. Additionally, externally built extensions can be uploaded in the UI.

Extension installation

In addition a table lists all installed analytics blocks with the following information: name, category, custom block, extension package name.

Block list

Manage custom extension

Custom extension can be uploaded from your local system. In addition they can be downloaded and deletes as well. For a deletion or upload to take effect you need to restart the analytics streaming engine.

Upload custom extension

An externally built extensions can be uploaded via the button Add extension. Simply drop the .zip file to the modal dialog and the extension will be loaded to the repository, but not yet deployed. To use them, restart the Streaming Analytics engine by clicking on the button Deploy extension (Restart) and wait for the notification confirming the engine restart. When you try to add an extension with a name that already exists, you have the choice to confirm that the version of the existing extension is replaces with the new one you are about to upload. You can as well cancel the process if this is not what you intend.

Build custom extension

After the deployment (restart of streaming analytics) the block will be available within the Streaming Analytics Application.

When the deployment of the extension was not successful an indicator Safe Mode modus appears in the action bar of the Manage extensions tab, as on the image below.

Monitoring

This usually happen when you try to deploy a faulty extension. In this case please check the logs of the microservice apama-ctrl-Xc-Xg, delete the extension and restart the Streaming Analytics engine.

Removing an extension will eliminate the block once more. It's important to note that when deleting an extension, there is no verification of whether the blocks within this extension are utilized in existing models. This could lead to models that are no longer deployable.

Use Extension

Build custom extension

You can build and uploads a custom extension by following the screen flow below:

Build custom extension

Options for custom extension

For a custom extension you have the following options:

Build custom extension

Samples repositories and building custom extensions

Block samples from GitHub repositories can be selected to build a custom extension online. To utilize this option, you must first configure your GitHub repository. Configured repositories can be updated, deleted, and enabled/disabled. Only enabled repositories will appear in the list of block samples. You can manage GitHub repositories using the provided user interface (UI):

Manage repositories

In addition you can view the EPL (event processing language) source code:

View source code

Building a custom extension starts by selecting the blocks from the list of samples. On selection of blocks an action in the table header appears custom extension. The modal dialog provides the option:

Create extension

Monitoring

On the monitoring tab you can view the latest alarms and events for the Streaming Analytics Engine. This is especially helpful if after an upload of a new extension no extension is loaded.

The property is_safe_mode is an indication if the engine was started in Safe Mode, i.e. without loading any extension. In this case you have to delete the latest uploaded extension and restart the engine again.

Monitoring

Installation of plugin as community plugin

NOTE This solution consists of two parts:

If the microservice is not deployed you can't build customs extensions.

The plugin is available as a community plugin and can be installed from the Administration -> Extensions -> Analytics Extensions:

Plugin installation

Build Instructions

This guide will teach you how to add the modified Cumulocity standard application, e.g. DeviceManagement, Cockpit, ... to your tenant.

Finally, you should see the new application in your App-Switcher.

Prerequisites to build plugin:

Instructions

Make sure you set the environments url, username, password before starting.

  1. Clone the repository:
    git clone https://github.com/SoftwareAG/cumulocity-analytics-management.git
  2. Change directory: cd cumulocity-analytics-management
  3. run npm i command to install all library files specified in source code npm i
  4. (Optional) Local development server: npm start
  5. Build the app: npm run build
  6. Deploy the app: npm run deploy

Analytics Builder Extension Backend

The microservice downloads the sample blocks from the configured repositories and builds an analytics extension as a zip file. This this zip file is downloaded locally. In an additional step it needs to be uploaded through UI, see Upload custom extension. You can specify if the extension should be uploaded automatically or it should be downloaded by the browser UI.

The microservice is multi tenant ready.

Prerequisites to build/deploy the microservice

Local debugging using Vscode/Devcontainer

To run and debug the microservice locally you an use Vscode and the .devcontainer/devcontainer.json configuration. To test with real c8y microservice credentials create an .env-admin (Administrative Credentials to fetch BootStrap Credentials) file in the analytics service directory and start the container (F1 -> Open in Open Folder in container). The get_service_creds.py fetch the credentials and create the .env file containing the bootstrap credentials.

To use your own credentials in the .env file comment the line "postAttachCommand": "python3 .devcontainer/get_service_creds.py", in devcontainer.json and create .env file with credentials of your liking. Env file format is:

After the dev container has started you can debug flask-wrapper.py as a python file.

To test the microservice you can issue the following curl command

curl --location 'http://127.0.0.1:<container host port>/extension' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic <tenant_id>/<username> as base64' \
--data '{
    "extension_name": "Test",
    "upload": true,
    "monitors": ["https://raw.githubusercontent.com/SoftwareAG/apama-analytics-builder-block-sdk/rel/10.18.0.x/samples/blocks/CreateEvent.mon"]
}'

Build/Deploy

To build the backend as a microservice analytics-ext-service follow these steps:

Analytics Builder block SDK

Custom blocks can be generated via the Analytics Builder block sdk. Find additional information on how blocks can be developed. However in the release section is one example block included that can be used for test purposes.

The provided block is an example of the adding the two inputs.

Use Extension

Troubleshooting

Note In order to check if an extension is deployed look for a relevant message in the log file of the analytics engine: [correlator] 2023-12-04 12:29:43.752 INFO [139659199286272] - Applying extension "/config/extensions/Sample_AB_Extension.zip"

The log file can be accessed: Administration> Ecosystem>Microservices>apama-ctrl-1c-4g>Logs

Note When a warning Microservice: analytics-ext-service not subscribed. Please subscribe this service before using the analytics plugin!appears please deploy the missing backend microservice.

Backend microservice missing


These tools are provided as-is and without warranty or support. They do not constitute part of the Software AG product suite. Users are free to use, fork and modify them, subject to the license agreement. While Software AG welcomes contributions, we cannot guarantee to include every contribution in the master project.


For more information you can Ask a Question in the TECHcommunity Forums.

You can find additional information in the Software AG TECHcommunity.