Card, svg elements specified with filter expressions not rendering

s2156945 commented 1 year ago

Describe the bug In V3 some elements do not show

Expected behavior elements are able to be ported from V2 JSON.

Screenshots

Example non-displaying SVG element config

woody@willie:~/work/git/meerkat/cmd/meerkat$ jq .elements[99] < dashboards/dab-overview.json 
{
  "options": {
    "audioSource": "",
    "backgroundColor": "",
    "boldText": false,
    "checkDataDefault": "",
    "checkDataPattern": "",
    "checkDataSelection": "",
    "checkId": "",
    "criticalAcknowledgedImage": "",
    "criticalAcknowledgedStrokeColor": "",
    "criticalImage": "",
    "criticalSound": "",
    "criticalStrokeColor": "#ff0019",
    "criticalSvg": "alert-octagon",
    "downSound": "",
    "dynamic": false,
    "dynamicText": "",
    "dynamicText2": "",
    "dynamicText2Structure": false,
    "filter": "\"Audio Services ABC Ultimo\" in service.groups&&match(\"*mux1A*\",host.__name)",
    "fontColor": "",
    "fontSize": 60,
    "image": "",
    "leftArrow": false,
    "linkURL": "https://icinga.onair.broadcastservices.cloud/icingaweb2/monitoring/list/services?servicegroup_name=Audio%20Services%20ABC%20Ultimo&sort=service_severity",
    "muteAlerts": false,
    "nameFontSize": 40,
    "objectName": "Audio Services ABC Ultimo",
    "objectType": "service",
    "okImage": "",
    "okSound": "",
    "okStrokeAcknowledgedColor": "",
    "okStrokeColor": "#0ee16a",
    "okSvg": "check-circle",
    "rightArrow": false,
    "scrollPeriod": "",
    "selection": "service-filter",
    "source": "",
    "strokeColor": "",
    "strokeWidth": 0,
    "svg": "",
    "text": "",
    "textAlign": "",
    "textVerticalAlign": "",
    "unknownAcknowledgedImage": "",
    "unknownAcknowledgedStrokeColor": "",
    "unknownImage": "",
    "unknownSound": "",
    "unknownStrokeColor": "#970ee1",
    "unknownSvg": "help-circle",
    "upSound": "",
    "warningAcknowledgedImage": "",
    "warningAcknowledgedStrokeColor": "",
    "warningImage": "",
    "warningSound": "",
    "warningStrokeColor": "#ff9000",
    "warningSvg": "alert-triangle"
  },
  "rect": {
    "h": 3.00001,
    "w": 3.00001,
    "x": 0,
    "y": 7.71971496437055
  },
  "rotation": 0,
  "title": "ABC Audio Source Ultimo A",
  "type": "check-svg"
}

Example non-displaying Card element

woody@willie:~/work/git/meerkat/cmd/meerkat$ jq .elements[35] < dashboards/dab-overview.json 
{
  "options": {
    "audioSource": "",
    "backgroundColor": "",
    "boldText": false,
    "checkDataDefault": "",
    "checkDataPattern": "",
    "checkDataSelection": "",
    "checkId": "",
    "criticalAcknowledgedImage": "",
    "criticalAcknowledgedStrokeColor": "",
    "criticalImage": "",
    "criticalSound": "",
    "criticalStrokeColor": "",
    "criticalSvg": "",
    "downSound": "",
    "dynamic": false,
    "dynamicText": "",
    "dynamicText2": "",
    "dynamicText2Structure": false,
    "filter": "\"site-abc-ultimo\" in service.groups && \"prj-dab\" in host.groups",
    "fontColor": "",
    "fontSize": 20,
    "image": "",
    "leftArrow": false,
    "linkURL": "https://icinga.onair.broadcastservices.cloud/icingaweb2/monitoring/tactical?hostgroup=prj-dab&modifyFilter=1",
    "muteAlerts": false,
    "nameFontSize": 40,
    "objectName": "site-abc-ultimo",
    "objectType": "service",
    "okImage": "",
    "okSound": "",
    "okStrokeAcknowledgedColor": "",
    "okStrokeColor": "",
    "okSvg": "",
    "rightArrow": false,
    "scrollPeriod": "",
    "selection": "service-filter",
    "source": "",
    "strokeColor": "",
    "strokeWidth": 0,
    "svg": "",
    "text": "",
    "textAlign": "",
    "textVerticalAlign": "",
    "unknownAcknowledgedImage": "",
    "unknownAcknowledgedStrokeColor": "",
    "unknownImage": "",
    "unknownSound": "",
    "unknownSvg": "",
    "upSound": "",
    "warningAcknowledgedImage": "",
    "warningAcknowledgedStrokeColor": "",
    "warningImage": "",
    "warningSound": "",
    "warningSvg": ""
  },
  "rect": {
    "h": 5.85215605749487,
    "w": 5.36951501154734,
    "x": 3.13961255845023,
    "y": 8.07600950118765
  },
  "rotation": 0,
  "title": "ABC Ultimo",
  "type": "check-card"
}
woody@willie:~/work/git/meerkat/cmd/meerkat$

ollytom commented 1 year ago

Ah this is an important behaviour I’ve addressed but haven’t written about properly yet. They will render by changing the way the object(s) are selected. I’ve made notes that I’ll put here tomorrow.

ollytom commented 1 year ago

I'm proposing to remove filter expressions from the interface, to instead be reference them by Service/Host group name.

Long story

I'll use the example elements from above. In the first element, this specified as the following expression:

"Audio Services ABC Ultimo" in service.groups && match("*mux1A", host.__name)

In the second element:

"site-abc-ultimo" in service.groups && "prj-dab" in host.groups

Those expressions don't have a name but you can sort of guess at what collection of objects the expression is selecting for. From the element's title; "Audio Services ABC Ultimo" in the first element and "ABC Ultimo" in the second.

Icinga provides an object type for this purpose: host and service groups.

The above expressions can be represented as a host/service group, specified by the same filter expression. For example:

apply ServiceGroup "abc-ultimo-mux1A" {
    display_name = "ABC Ultimo Mux1A"
    assign where "Audio Services ABC Ultimo" in service.groups && match("*mux1A", host.name)
}

I'd argue that giving a name to the group is clearer than using a raw expression. Compare "men" to SELECT * WHERE gender="male".

Now that we've named the thing we were referring to all along, it can be queried from the API, Icingaweb, OpsGenie, notifications...

For Meerkat, you remove the freeform text field and the responsibilities of having such a feature re: documentation, error handling. Selecting a group is already done; you just click from a dropdown box.

Freeform text use case

I can imagine a use case where writing filter expressions and seeing some box change colour as a way to kind of evaluate the validity of an expression. For exploratory use cases, such as evaluating the results of some expressions, a basic command-line tool or minimal web app would have these bonuses over Meerkat:

more suitable - not a little narrow text field
more portable - generic icinga ecosystem utility
more fun to maintain! - writing small tools is always more fun ;)

I'm all for "little languages" AKA domain-specific languages. I would love to share how Unix and OpenBSD and Plan 9 all took advantage of them. And then there's those projects wrestling with ORMs over using plain SQL!

Summary

I think we can save a feature, dev budget, maintenance by sticking with only exposing the names of the groups - which themselves are defined using filter expressions.

Consequences

Not implementing this feature means that groups will have to be made from the expressions specified in dashboards. From an overall operational viewpoint I anticipate a net-win since those groups can be used for notifications and in icingaweb dashboards They could even have a little documentation attached as a comment.

ollytom commented 1 year ago

We discussed this and did an analysis of approx. 5000 elements which used a filter expression, rather than a name, to specify an icinga object. Approx. 50% can be replaced by a reference by name. But the other 50% would not be supported. So we worked out that we need to expose filter expressions in the UI.

One common use case is to create an element which represents the state of all services on a particular host. It's impractical to maintain ServiceGroups for each Host; there may be thousands to tens of thousands.

ollytom commented 1 year ago

I’ve got this for line and svg icon elements. I haven’t figured out how it will work for card elements. Not yet sure how to represent all the attributes from each Icinga object.

For groups, we provide 2 attributes: the worst state and soonest check command of all objects in the group. We can just use all the same logic as it applies to a list of objects, which filter expressions return.

The ability to use the state attribute from one object (or objects), then render the text of an attribute from a different object would be covered by #148.

In that case I could submit the code I’ve got as is, close this issue, and leave the remaining to be done in #148

meerkat-dashboard / meerkat