showHost option for icingadb

[gbin2265]

@nbuchwitz @sukhwinder33445

Hello,

Describe the bug

In icingaweb there is a link with the parameter 'showHost='. This option will first request the complete list of all hosts (6k in my case), create all points on the map and then filter only the host you request. (in my situation this takes 8 minutes)

If you would do the same by directly filtering the 'host.name=', it will first apply the filter to the database and then create the points on the map , and immediately I have the host on the map.

Possible Solution

Who should I ask to optimize this?

I create a case in icingaweb-db to replace the showhost with host.name or can you put an extra filter in the Data Controller that converts the showHost to host.name for icingadb ?

regards, Geert

[nbuchwitz]

Hi Geert,

it would be interesting what is mostly contributes to the 8 minutes load time. The frontend, backend or database? Imho fetching 6k rows should not take 8 minutes.

The showHost loads the map the normal way and then opens the referenced host object. This will allow to use the map as usual in the same view (eg. further analysis of an outage).

[gbin2265]

Hello,

I am trying to collect information in the program to know how long each action takes.

Now I see that the Data Controller restarts every 2 minutes. Is there a timer somewhere that says the map needs to be refreshed every 2min ?

So that the processing of the 200k services is more than 2 minutes, and after 2 minutes a 2nd session starts, and then 2 minutes later a 3rd session... And so one ends up in the situation that the host cannot follow to create the first map.....

regards, Geert

[gbin2265]

Hello,

I had to adjust parameters so that PHP has the chance to create the map for my 6k hosts once :

Parameters

cat /etc/httpd/conf/httpd.conf TimeOut 600

/etc/opt/rh/rh-php73/php.ini max_execution_time = 900 max_input_time = 900 memory_limit = 4096M

Results

It takes 40 sec to read the 200k lines in the database.

This loop takes about 7 minutes :

        foreach ($hostQuery as $row) {
            if (! preg_match($this->coordinatePattern, $row->vars['geolocation'])) {
                continue;
            }

            $hostname = $row->name;
            if (! isset($this->points['hosts'][$hostname])) {
                $host = $this->populateObjectColumnsToArray($row);
                $host['host_problem']               = $row->state->is_problem ? 1 : 0;
                $host['coordinates']                = $row->vars['geolocation'];
                $host['icon']                       = $row->vars['map_icon'] ?? null;
                $host['coordinates'] = explode(",", $host['coordinates']);

                $host['services'] = [];

                $this->points['hosts'][$row->name] = $host;
            }

        if ($row->service->id !== null) {
            $service = $this->populateObjectColumnsToArray($row->service);
            $this->points['hosts'][$hostname]['services'][$row->service->display_name] = $service;
        }
    }

Multiple sessions

but it could be because the session restarts every 2 minutes that the first session takes much more time because there are multiple processes running and running ...

[nbuchwitz]

Thanks this helps a lot. I will try to reduce the loop runtime. In the meantime you can adjust the timer rate here https://github.com/nbuchwitz/icingaweb2-module-map/blob/master/public/js/module.js#L244. The value is in milliseconds, thus 60000 renders to 60 seconds = 1 minute refresh rate.

[gbin2265]

Hello nbuchwitz,

I remove everything from the foreach except this

       foreach ($hostQuery as $row) {

            $hostname = $row->name;
            $teller = $teller + 1;
            }

After 7 minutes I get the result of teller -> 229381 (number of records)

I see in the monitoring that one virtual processor does everything and the other does nothing.

I don't think you can improve anything at this point. I will have to see in the area of configuration php/fpm/... that I can improve the speed.

I'll see if I can't create an extra option (such as objectType=hostonly) where I only request the hosts (6k) from the db and show them on the map You will not see the services in the map module, but you can view them in the details screen of the host. (I'm not a PHP programmer, so I'll have to do some searching)

In any case, thanks for the support!

regards, Geert

[nbuchwitz]

That's ~550 rows per second. Even if I'm a big fan of php-fpm I don't think that they can increase the speed here. The loop cannot be split in to different worker tasks which could run on another CPU core. Another thought: Is the database server running on spinning disks or flash? Maybe your DBA can have a look where the database server spends time on. Sometimes the right combination of an index can work miracles.

[gbin2265]

Transfert of the 200k from the db to the memory takes 40 sec. It might gain a few more seconds with other indexes, but that won't make much of a difference in the 7 minutes it needs in memory to put everything on the map. Parameter memory_limit = 4096M of the 24G vm memory. No swapping of memory.

I think we're just at the limit of php.

[nbuchwitz]

Hm ok. So the 7 minutes are not even map processing code, but mostly processing of icinga db results (your simplified loop took also 7 minutes).

The best solution would be to do some lazy loading and only request the host / service data for visible elements. But this would be a larger project for which I don't have time at the moment :(

[tectumopticum]

I'm not sure if my problem has the same root-cause. We're on "icingadb-only", which means the monitoring-module is disabled and the IDO-DB has been dropped. Since then klicking on the "Show on map"-link (https:///icingaweb2/map?showHost=hugo.fq.dn) shows the default-map immediately without any object. About 50 seconds later you see the circles of the objects in map but not a zoomed-view of the object I chose.

[gbin2265]

Hello @tectumopticum

My configuration is also completely icingadb/icingadb-web (ido completely disabled).

On my production (6k objects, 230k services) everything works perfectly. After 6 minutes I get the map with the focus on the host.

To prevent it from doing a refresh, I set the refresh from 60000 to 0. I modify 60000 to 0 on the line 244 in the file public/js/module.js So there is no refresh of the map after 60sec.

I don't know if this info helps.

[tectumopticum]

Hello @tectumopticum

My configuration is also completely icingadb/icingadb-web (ido completely disabled).

On my production (6k objects, 230k services) everything works perfectly. After 6 minutes I get the map with the focus on the host.

To prevent it from doing a refresh, I set the refresh from 60000 to 0. I modify 60000 to 0 on the line 244 in the file public/js/module.js So there is no refresh of the map after 60sec.

I don't know if this info helps.

@nbuchwitz @gbin2265 Hi Geert, hi Nicolai thanks for your hints - unfortunately this doesn't improve anything. Since changing to icingadb the performance of the map-module is poor. We have ~3.2k hosts, 22.6k services. I also have some maps in a dashboard as dashlets - something like https://icingahost/icingaweb2/map?_host_facility=test&default_lat=&default_long=&default_zoom=16&problems Now I see nothing else than an empty background. I remember when I created this view and opened the dashboard with the three dashlets it was filled instantaneously with the maps and their hosts. As a workaround I can click on the dashlet's title (the link to the query). As a result the map opens but no host-objects will appear.

[gbin2265]

Hello,

Shouldn't you modify your link as it is now for icingadb-web?

Before : _host_facility=test

Now : host.vars.facility=test

[gbin2265]

I have no experience in the PHP programming language, and it is difficult to judge why it does not go that fast.

But I'm glad I'm not the only one with the feeling that it's so slow

Reading from the database (especially after the last modification) is fast for me, but it is the step of creating the MAP that is so slow.

II did a simple test in the file application/controllers/DataController.php

After the line 283

$hostQuery = $hostQuery->execute();
if (! $hostQuery->hasResult()) {
            return;
        }

I do a loop of all objects without any action and this just takes so much time

 foreach ($hostQuery as $row) {

           $hostname = $row->name;
           $teller = $teller + 1;
           }

I have the impression that there is so much information in the hostQuery that it just makes the loop so slow.

But then again, I'm not a PHP programmer

I leave this more to PHP specialists such as @nbuchwitz @sukhwinder33445