task: Add ability to run incremental pipeline for Python Checks

Component

API Server / GraphQL, Git Integration

Task Description

Currently this is possible for artifact_definitions, it should be possible to do with check_definitions too. Two differences:

Lacking alternative to "artifacts"

The artifact_definitions currently has a artifacts that are subscribers to GraphQLQueryGroups. For check_definitions we don't have such an object to track in order to determine if check_definitions might be impacted due to which nodes in the graph has been changed.

It would be possible to analyze the GraphQL queries and base it on the impacted models as an initial state.

A Check definition can be global

Currently artifact_definitions are always tied to a target group where as a check_definition could also be of a global type to run a check on everything. We need to consider the implication on this when determining the type of "check" object to create as a target for a global check.

As a note. It could also be that we should support "global" artifact_definition in the same way which could be used to create various reports.

I started looking at this and created a schema object looking like this:

        {
            "name": "GeneratorInstance",
            "namespace": "Core",
            "label": "Generator Instance",
            "include_in_menu": False,
            "icon": "mdi:file-document-outline",
            "default_filter": "name__value",
            "order_by": ["name__value"],
            "display_labels": ["name__value"],
            "branch": BranchSupportType.LOCAL.value,
            "inherit_from": [InfrahubKind.TASKTARGET],
            "attributes": [
                {"name": "name", "kind": "Text"},
                {
                    "name": "status",
                    "kind": "Text",
                    "enum": GeneratorInstanceStatus.available_types(),
                },
            ],
            "relationships": [
                {
                    "name": "object",
                    "peer": InfrahubKind.NODE,
                    "kind": "Attribute",
                    "identifier": "generator__node",
                    "cardinality": "one",
                    "optional": False,
                },
                {
                    "name": "definition",
                    "peer": InfrahubKind.GENERATORDEFINITION,
                    "kind": "Attribute",
                    "identifier": "generator__generator_definition",
                    "cardinality": "one",
                    "optional": False,
                },
            ],
        },

The idea would be that the object relationship would point to a group member for "targeted" checks and to the "CoreCheckDefinition" itself for "global" checks.

Following this logic the checks could work in the same way as current artifact_definitions where the objects impacted by the checks would be subscribers, or the check_definition would be the subscriber for a "global" check. However it could be that we in some situations want to always trigger a PythonCheck to run even though its query didn't contain anything that changed. Examples of this could be if the purpose of the check itself is to reach out to an external resource such as ServiceNow or similar.

For this we might want a new attribute on CoreCheckDefinition to control this behaviour, i.e. if the checks should run even though no changes were made to any impacted objects. The alternative that I see would be to always run the "global" checks. But I think the option to control it would be better.

Any thoughts around this?

opsmill / infrahub