search

Dynatrace / backstage-plugin

Dynatrace Backstage Plugin
Apache License 2.0
21 stars 3 forks source link

feat: custom DQL queries within catalog-info.yaml #124

Closed blasget closed 3 months ago

blasget commented 4 months ago

Introduction

resolves #46

Technical Solution Before This PR

Previously, custom DQL queries had to be defined within the app-config.yaml file of the Backstage instance. This approach did not allow developers to define their own custom actions for their components. Moreover, for each custom query, an EntityDqlQueryCard with the query ID had to be added.

Technical Solution After This PR

The EntityCatalogInfoQueryCard is used to display a result table for each query defined in the spec.queries property of an entity in the catalog-info.yaml file. This eliminates the need to provide query IDs.

To showcase custom queries, an additional route was added to the EntityPage.tsx file in the demo-backstage component.

Additionally, three sample queries were added to the demo-backstage component:

image

:heavy_check_mark: Common checklist

blasget commented 4 months ago

Updated PR

Introduction

resolves #46 and resolves #107

Technical Solution Before This PR

Previously, custom DQL queries had to be defined within the app-config.yaml file of the Backstage instance. This approach did not allow developers to define their own custom actions for their components. Moreover, for each custom query, an EntityDqlQueryCard with the query ID had to be added.

There was no default query to get site reliability guardian validations and display them in Backstage.

Technical Solution After This PR

The Entity interface doesn’t support adding custom properties such as dynatrace.queries. As a result, the decision was made to use annotations and Dynatrace Notebooks instead. This method offers a significant benefit: it enables users to create and test their custom queries directly within their Dynatrace environment. These queries can be included in a backstage entity via annotations defined in the catalog-info.yaml file. There are two types of notebook annotations supported:

  1. dynatrace.com/notebook-url: https://my-environment.dynatrace.com/ui/apps/dynatrace.notebooks/notebook/my-notebook-id
  2. dynatrace.com/notebook-id: my-notebook-id

image

The EntityNotebookQueriesCard is used to display a result table for each query defined in a Dynatrace Notebook. Placeholders are not supported.

To showcase custom queries, an additional route was added to the EntityPage.tsx file in the demo-backstage component.

Additionally, three sample queries were added to the README.md:

A default query has been integrated to automatically fetch SRG validation results from Dynatrace. Each entry in the validation results table now includes a direct link that opens the Site Reliability Guardian (SRG) view.

image

To filter for specific guardians, you can filter tags defined in the metadata.annotations property of the catalog-info.yaml file under the key dynatrace.com/guardian-tags.

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: demo-backstage
  description: Backstage Demo instance.
  annotations:
    backstage.io/kubernetes-id: kubernetescustom
    dynatrace.com/guardian-tags: 'service=my-service,stage=development,novalue'

There are two ways to filter tags:

  1. Key-Value Match: The tag must match both the key and the value. For example, the key service must have the value my-service.
  2. Key Exists: The tag key must exist with any value. For example, the key novalue.

The difference between these filtering methods is indicated by the presence of the = symbol.

:heavy_check_mark: Common checklist

blasget commented 3 months ago

First of all great work!

I would also have some questions :)

  • Which timeframe is taken for the Notebook DQLs?
  • Is there a need to share a Notebook in order to be accessible by the E2E client? E.g., userA creates Notebook1, can the client read it or does it need to be shared?
  • Fetching from Notebooks probably also needs another permission right? I would assume document:documents:read. Could you add it to the readme?
  1. Currently, only timeframes specified within the query are supported.
  2. The notebook must be shared, otherwise the client will receive "code":403,"message":"Document not accessible: document-id
  3. To read the document, this permission is required. I will update the README.md.
johannes-b commented 3 months ago

In a review session of the current status, we found a problem with the current implementation of referencing a Notebook.

The problem is the dependency on an external resource (Notebook) when scaling out the functionality leveraging, e.g., Backstage templates. For instance:

While we agreed it is a developer-friendly approach to referencing (my) notebook, the scaling issue(s) changed our view on the topic. After revisiting the Backstage extensibility, we discovered that metadata property allows custom properties. See: https://backstage.io/docs/features/software-catalog/extending-the-model/#adding-new-fields-to-the-metadata-object

Based on that, we derived the following approach:

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: demo-backstage
  description: Backstage Demo instance.
  annotations:
    backstage.io/kubernetes-id: kubernetescustom
    dynatrace.com/guardian-tags: "service=my-service,stage=development,novalue"
  dynatrace:
    queries:
      - id: davis-events-1
        description: Fetches all Davis events
        query: >
          fetch events | filter event.kind == "DAVIS_EVENT" | fields event.kind, timestamp
      - id: biz-events-2
        description: Fetches some BIZ_EVENTS
        query: >
          fetch bizevents | filter event.kind == "BIZ_EVENTS" | fields event.kind, timestamp
spec:
  type: website
  owner: user:default
  lifecycle: experimental
  system: integrations

Given this example, it is possible to define multiple custom queries in the config-info.yaml that are executed against all configured Dynatrace environments. The result of one query is then merged into one table, which the plugin displays.