Icinga / icinga2

The core of our monitoring platform with a powerful configuration language and REST API.
https://icinga.com/docs/icinga2/latest
GNU General Public License v2.0
2.03k stars 578 forks source link

No object type for redis key icinga:dependency:customvar found #9478

Closed mzac closed 2 years ago

mzac commented 2 years ago

Describe the bug

Icingadb daemon is crashing on start

2022-08-09T10:43:24.961-0400    INFO    icingadb        Starting Icinga DB
2022-08-09T10:43:24.961-0400    INFO    icingadb        Connecting to database at 'localhost:3306'
2022-08-09T10:43:24.967-0400    INFO    icingadb        Connecting to Redis at 'localhost:6380'
2022-08-09T10:43:24.968-0400    INFO    icingadb        Starting history sync
2022-08-09T10:43:25.853-0400    INFO    heartbeat       Received Icinga heartbeat       {"environment": "612491695a81d0cfb21c30d8e08ef32d631757e6"}
2022-08-09T10:43:25.862-0400    INFO    icingadb        Taking over
2022-08-09T10:43:25.869-0400    INFO    icingadb        Starting config sync
2022-08-09T10:43:25.869-0400    INFO    icingadb        Starting initial state sync
2022-08-09T10:43:25.869-0400    INFO    icingadb        Starting overdue sync
2022-08-09T10:43:26.182-0400    INFO    config-sync     Updating 8802 items of type host state
2022-08-09T10:43:26.389-0400    INFO    config-sync     Updating 21610 items of type service state
2022-08-09T10:43:27.579-0400    INFO    config-sync     Finished initial state sync in 1.710440468s
2022-08-09T10:43:27.579-0400    INFO    icingadb        Starting state runtime updates sync
2022-08-09T10:43:28.057-0400    INFO    config-sync     Inserting 981 items of type host customvar
2022-08-09T10:43:28.057-0400    INFO    config-sync     Deleting 980 items of type host customvar
2022-08-09T10:43:28.105-0400    INFO    icingadb        Starting history retention
2022-08-09T10:43:28.105-0400    INFO    icingadb        Starting config runtime updates sync
2022-08-09T10:43:28.105-0400    INFO    config-sync     Finished config sync in 2.236165018s
2022-08-09T10:43:28.122-0400    FATAL   icingadb        no object type for redis key icinga:dependency:customvar found
github.com/icinga/icingadb/pkg/icingadb.(*RuntimeUpdates).xRead.func1
        github.com/icinga/icingadb/pkg/icingadb/runtime_updates.go:262
golang.org/x/sync/errgroup.(*Group).Go.func1
        golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57
runtime.goexit
        runtime/asm_amd64.s:1571

To Reproduce

Apply custom vars to dependencies is what I'm guessing is crashing it

Expected behavior

It should not crash and should accept custom vars in dependencies

Your Environment

Include as many relevant details about the environment you experienced the problem in

Additional context

An example of a decency we add via the API:

[root@icinga01 dependencies]# more newres-3-314-ap1\!newres-245-sw1.conf
object Dependency "newres-245-sw1" {
        child_host_name = "newres-3-314-ap1"
        parent_host_name = "newres-245-sw1"
        vars = {
                database_run = "Thu Jul 28 00:04:00 2022"
                database_source = "cmdb"
                device_type = "aruba_ap"
        }
        version = 1658982689.918957
        zone = "icinga01"
}
Al2Klimov commented 2 years ago

Hello,

what are the custom vars for?

Best, A/K

mzac commented 2 years ago

@Al2Klimov We use vars in our dependencies so we are able to delete old ones as we update our Icinga instance every 2 hours and we have multiple CMDB sources to poll.

When we insert the dependency, the vars contain the CMDB source, the device type and the time it we added. When the script re-runs it updates this information and then deletes any dependencies that don't match the new timestamp.

This is an example from one of our Perl scripts on how we delete old entries:

    # --------------------------------------------------------------------------------
    # Delete old dependencies
    my %json_delete_data_dependency = (
        'filter' => "match(\"". $icinga_device_type ."\",dependency.vars.device_type) && match(\"". $database_source ."\",dependency.vars.database_source) && !match(\"". $datestring ."\",dependency.vars.database_run)",
    );

    my $delete_data_dependency = encode_json(\%json_delete_data_dependency);
    $delete_client->POST("$icinga_rest_base_url/dependencies\?cascade=1", $delete_data_dependency);