search

objectiveit / zabbix-kubernetes

GNU General Public License v3.0
19 stars 4 forks source link

Zabbix API HTTP Connection Error at /usr/lib/zabbix/externalscripts/k8s_collect.pl line 65 #1

Open ugoviti opened 5 years ago

ugoviti commented 5 years ago

Hi,

very nice project!

The k8s_discovery.pl works good, but I have a problem with k8s_collect.pl

I double checked the user and password API, but it doesn't works:

Zabbix API HTTP Connection Error at /usr/lib/zabbix/externalscripts/k8s_collect.pl line 65.

this URL is correct: my $APIURL = 'http://127.0.0.1/zabbix/api_jsonrpc.php';

and logging in the gui with the specified APIUSER and APIPASS works without problems (the specified user is Super Admin with read/write access to all Zabbix hostgroups).

How can I debug the Zabbix API HTTP Connection Error?

I'm using Zabbix 4.2.1

Kind regards

Frintrop commented 5 years ago

Maybe zabbix is not running on local interface ?

Can u curl http://127..0.0.1/zabbix/ ?

Von: Ugo Viti [mailto:notifications@github.com] Gesendet: Montag, 29. April 2019 23:05 An: objectiveit/zabbix-kubernetes zabbix-kubernetes@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Betreff: [objectiveit/zabbix-kubernetes] Zabbix API HTTP Connection Error at /usr/lib/zabbix/externalscripts/k8s_collect.pl line 65 (#1)

Hi,

very nice project!

The k8s_discovery.pl works good, but I have a problem with k8s_collect.pl

I double checked the user and password API, but it doesn't works:

Zabbix API HTTP Connection Error at /usr/lib/zabbix/externalscripts/k8s_collect.pl line 65.

this URL is correct: my $APIURL = 'http://127.0.0.1/zabbix/api_jsonrpc.php';

and logging in the gui with the specified APIUSER and APIPASS works without problems (the specified user is Super Admin with read/write access to all Zabbix hostgroups).

How can I debug the Zabbix API HTTP Connection Error?

I'm using Zabbix 4.2.1

Kind regards

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/objectiveit/zabbix-kubernetes/issues/1, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABP63MVMVJ7VED62NWWOF5TPS5PIHANCNFSM4HJGX2SQ.

ugoviti commented 5 years ago

Hi,

thank you for the reply.

Yes, Zabbix web server is listening on local loopback:

[zabbix@mon01 ~]$ curl http://127.0.0.1/zabbix/
<!DOCTYPE html>
<html>
    <head>
        <meta http-equiv="X-UA-Compatible" content="IE=Edge"/>
        <meta charset="utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <meta name="Author" content="Zabbix SIA" />
        <title>izMon: Zabbix</title>
        <link rel="icon" href="favicon.ico">

same result using http://localhost/zabbix/

running by hand I got the following error:

[zabbix@mon01 ~]$ /usr/lib/zabbix/externalscripts/k8s_collect.pl 127.0.0.1 /var/lib/zabbix/.kube/config test
Zabbix API HTTP Connection Error at /usr/lib/zabbix/externalscripts/k8s_collect.pl line 65.
[root@mon01 ~]# netstat -atpun | grep zabbix
tcp        0      0 0.0.0.0:10050           0.0.0.0:*               LISTEN      17605/zabbix_agentd 
tcp        0      0 0.0.0.0:10051           0.0.0.0:*               LISTEN      17690/zabbix_server 
tcp6       0      0 :::10050                :::*                    LISTEN      17605/zabbix_agentd 
tcp6       0      0 :::10051                :::*                    LISTEN      17690/zabbix_server

zabbix-kubernetes is compatible with Zabbix 4.2? Can I look into or debug other options? How can I debug the API accesso from python?

Thank you for the help, keep up the good work.

Kind regards

Frintrop commented 5 years ago

Collect script is configured inside. Also default agent port ... ZABBIX_PORT = '10051';

It uses not api.... it sends to agent.

Von unterwegs gesendet.

Am 03.05.2019 um 17:29 schrieb Ugo Viti notifications@github.com<mailto:notifications@github.com>:

Hi,

thank you for the reply.

Yes, Zabbix web server is listening on local loopback:

[zabbix@mon01 ~]$ curl http://127.0.0.1/zabbix/ <!DOCTYPE html>

izMon: Zabbix same result using http://localhost/zabbix/ running by hand I got the following error: [zabbix@mon01 ~]$ /usr/lib/zabbix/externalscripts/k8s_collect.pl 127.0.0.1 /var/lib/zabbix/.kube/config test Zabbix API HTTP Connection Error at /usr/lib/zabbix/externalscripts/k8s_collect.pl line 65. [root@mon01 ~]# netstat -atpun | grep zabbix tcp 0 0 0.0.0.0:10050 0.0.0.0:* LISTEN 17605/zabbix_agentd tcp 0 0 0.0.0.0:10051 0.0.0.0:* LISTEN 17690/zabbix_server tcp6 0 0 :::10050 :::* LISTEN 17605/zabbix_agentd tcp6 0 0 :::10051 :::* LISTEN 17690/zabbix_server zabbix-kubernetes is compatible with Zabbix 4.2? Can I look into or debug other options? How can I debug the API accesso from python? Thank you for the help, keep up the good work. Kind regards — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
MogiePete commented 5 years ago

@Frintrop

I am experiencing the same issue. I am running 4.0

Frintrop commented 5 years ago

We are running zabbix 4.0.7 and changed only api user and password in that script. Discovery is working ?

Api user has admin role in zabix.

Can u print out json result of request ?

Von: Mogie Pete notifications@github.com Gesendet: Freitag, 3. Mai 2019 20:55 An: objectiveit/zabbix-kubernetes zabbix-kubernetes@noreply.github.com Cc: André Frintrop andre.frintrop@objectiveit.de; Mention mention@noreply.github.com Betreff: Re: [objectiveit/zabbix-kubernetes] Zabbix API HTTP Connection Error at /usr/lib/zabbix/externalscripts/k8s_collect.pl line 65 (#1)

@Frintrophttps://github.com/Frintrop

I am experiencing the same issue. I am running 4.0

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/objectiveit/zabbix-kubernetes/issues/1#issuecomment-489203220, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABP63MU2Y3WUSVNOYGOPFGDPTSC7JANCNFSM4HJGX2SQ.

MogiePete commented 5 years ago

Discovery is working.

Using the code provided by Zabbix I am able to authenticate with the account created to monitor k8s so the account is working and the server is taking connections.

Getting started with Zabbix API

[root@svrzbx01 ~]# cat auth.pl
#!/usr/bin/perl

use 5.010;
use strict;
use warnings;
use JSON::RPC::Legacy::Client;
use Data::Dumper;

# Authenticate yourself
my $client = new JSON::RPC::Legacy::Client;
my $url = 'http://127.0.0.1/zabbix/api_jsonrpc.php';
my $authID;
my $response;

my $json = {
jsonrpc => "2.0",
method => "user.login",
params => {
user => "zabbixapi",
password => "zabbixapipass"
},
id => 1
};

$response = $client->call($url, $json);

# Check if response was successful
die "Authentication failed\n" unless $response->content->{'result'};

$authID = $response->content->{'result'};
print "Authentication successful. Auth ID: " . $authID . "\n";
[root@svrzbx01 ~]# ./auth.pl
Authentication successful. Auth ID: 121cd383ee6a40c6d9589fea5b3b7b75

And here is the code if I try to log in using curl.

 [root@zabbixserver ~]# curl -i -X POST -H 'Content-type:application/json' -d '{"jsonrpc":"2.0","method":"user.login","params":{ "user":"zabbixapi","password":"zabbixapipass"},"auth":null,"id":0}' http://127.0.0.1/zabbix/api_jsonrpc.php
 HTTP/1.1 200 OK
 Date: Sun, 05 May 2019 06:53:09 GMT
 Server: Apache/2.4.6 (Red Hat Enterprise Linux) PHP/5.4.16
 X-Powered-By: PHP/5.4.16
 Access-Control-Allow-Origin: *
 Access-Control-Allow-Headers: Content-Type
 Access-Control-Allow-Methods: POST
 Access-Control-Max-Age: 1000
 Content-Length: 68
 Content-Type: application/json

{"jsonrpc":"2.0","result":"ea54cb61db9a7d0a864565dd6a84c53d","id":0}
mfocuz commented 5 years ago

Hi All,

Please pay attention that discovery do not use Zabbix API, so there is no "dependency" between "discovery works" and "collector doesnt work".

Your issue with API connection most probbaly related to Zabbix API User. If username and password is correct and URL "localhost" is accessible from script(that means you run script from zabbix WEB server) - THEN YOU NEED TO CHECK THE FOLLOWING:

  1. Zabbix User has access to API
  2. Zabbix API User has at least ADMIN ROLE. Better to set SUPER ADMIN for troubleshooting, after you verify that script works correctly, reduce role to ADMIN. (I'm 99% sure its your case root cause).

ADMIN permissions required for reading item's keys.

Maxim

ugoviti commented 5 years ago

Hi, I double checked the user password and privileges (testing as Super Admin right now).

And tested with script auth.pl of @MogiePete and it's works good using the same url user and password of k8s_collect.pl:

$ ./auth.pl
Authentication successful. Auth ID: 867559bdf856c121dba44bad6b413795

I'm running it into a CentOS 7 system with official perl modules installed.

$ rpm -qa | grep -e LWP -e JSON
perl-LWP-MediaTypes-6.02-2.el7.noarch
perl-LWP-Protocol-https-6.04-4.el7.noarch
perl-JSON-RPC-1.03-5.el7.noarch
perl-JSON-2.59-2.el7.noarch

What perl version are you using?

Kind regards

mfocuz commented 5 years ago

Please try to add response printing adter line 59 with: print Dumper($respAuth->content); and uncomment use Dumper; on line 6

Its very rare case when perl libs versions are root cause, also I dont remember versions I checked. If you have successfully logged in with custom API script, all you need to troubleshoot is to find HTTP request differences between you custom script and script in this repo. Any HTTP proxy you have exp with will help.

ugoviti commented 5 years ago

Thankyou, made the change, now I got:

$ /usr/lib/zabbix/externalscripts/k8s_collect.pl 127.0.0.1 /var/lib/zabbix/.kube/config test
Global symbol "$response" requires explicit package name at /usr/lib/zabbix/externalscripts/k8s_collect.pl line 61.
Execution of /usr/lib/zabbix/externalscripts/k8s_collect.pl aborted due to compilation errors.
mfocuz commented 5 years ago

please check again, I've edited comment in few secodns, you read too fast :D print Dumper($respAuth->content);

MogiePete commented 5 years ago

@mfocuz

I have it dump the $respAuth and the content and this is the results:

$VAR1 = bless( {
                 '_msg' => 'Precondition Failed',
                 '_content' => '',
                 '_protocol' => 'HTTP/1.0',
                 '_headers' => bless( {
                                        'connection' => 'close',
                                        'access-control-allow-headers' => 'Content-Type',
                                        'x-powered-by' => 'PHP/5.4.16',
                                        'client-response-num' => 1,
                                        'access-control-allow-origin' => '*',
                                        'date' => 'Sun, 05 May 2019 16:09:11 GMT',
                                        'access-control-allow-methods' => 'POST',
                                        'client-peer' => '127.0.0.1:80',
                                        'content-length' => '0',
                                        '::std_case' => {
                                                          'access-control-allow-headers' => 'Access-Control-Allow-Headers',
                                                          'x-powered-by' => 'X-Powered-By',
                                                          'client-response-num' => 'Client-Response-Num',
                                                          'access-control-allow-origin' => 'Access-Control-Allow-Origin',
                                                          'access-control-allow-methods' => 'Access-Control-Allow-Methods',
                                                          'client-peer' => 'Client-Peer',
                                                          'client-date' => 'Client-Date',
                                                          'access-control-max-age' => 'Access-Control-Max-Age'
                                                        },
                                        'client-date' => 'Sun, 05 May 2019 16:09:11 GMT',
                                        'content-type' => 'text/html; charset=UTF-8',
                                        'access-control-max-age' => '1000',
                                        'server' => 'Apache/2.4.6 (Red Hat Enterprise Linux) PHP/5.4.16 mod_perl/2.0.10 Perl/v5.16.3'
                                      }, 'HTTP::Headers' ),
                 '_rc' => '412',
                 '_request' => bless( {
                                        '_content' => '{"params":{"password":"zabbixapipass","user":"zabbixapi"},"jsonrpc":"2.0","id":1,"method":"user.login"}',
                                        '_uri' => bless( do{\(my $o = 'http://127.0.0.1/zabbix/api_jsonrpc.php')}, 'URI::http' ),
                                        '_headers' => bless( {
                                                               'user-agent' => 'libwww-perl/6.05',
                                                               'content-type' => 'application/x-www-form-urlencoded',
                                                               'content-length' => 103
                                                             }, 'HTTP::Headers' ),
                                        '_method' => 'POST',
                                        '_uri_canonical' => $VAR1->{'_request'}{'_uri'}
                                      }, 'HTTP::Request' )
               }, 'HTTP::Response' );
$VAR1 = '';
Zabbix API HTTP Connection Error at ./k8s_collect.pl line 66.
mfocuz commented 5 years ago

you have HTTP response code 412, no idea why. I recommend to use HTTP proxy or tcpdump to verify difference between 2 HTTP requests.

posuch commented 5 years ago

got the same thing - zabbix 4.0.x on centos 7, anybody was able to fix it ?

posuch commented 5 years ago

so I guess what helped me with the authentification error is that in collect script in config section the IPs/hosts needed to be the same after that the script can connect to the API

see IP in $APIURL and $ZABBIXSERVER

CONFIG

my $APIURL = 'http://10.0.2.15/zabbix/api_jsonrpc.php'; my $APIUSER = 'api'; my $APIPASS = 'api'; my $ZABBIX_SENDER = '/usr/bin/zabbix_sender'; my $ZABBIX_SERVER = '10.0.2.15'; my $ZABBIX_PORT = '10051'; my $KUBECTL = '/usr/bin/kubectl'; my $TMP_FILE_PATH = '/tmp/send_tozabbix'. int(rand(1000000000)) .'.data';

the thing is now that the data are not comming to zabbix only if I run it by hand than some data comes in to it ( can anybody repeat this ? )

from the zabbix log ( no data sent ) in lastest data still zabbix api connection error

18969:20190812:091944.651 End of substitute_key_macros():SUCCEED data:'k8s_collect.pl[10.0.2.15,/root/.minikube/config/config.json,kube-system]' 18969:20190812:091944.651 In get_value_external() key:'k8s_collect.pl[10.0.2.15,/root/.minikube/config/config.json,kube-system]' 18969:20190812:091944.651 In zbx_popen() command:'/usr/lib/zabbix/externalscripts/k8s_collect.pl '10.0.2.15' '/root/.minikube/config/config.json' 'kube-system''

zabbix_api_error

but if I run this command by hand I get at least some data in the console ( works if run as root )

[root@centos7 externalscripts]# /usr/lib/zabbix/externalscripts/k8s_collect.pl '10.0.2.15' '/root/.minikube/config/config.json' 'kube-system' info from server: "processed: 46; failed: 0; total: 46; seconds spent: 0.002099" sent: 46; skipped: 0; total: 46

ninjageek-zcs commented 5 years ago

It worked for me after using JSON :: RPC :: Legacy :: Client for both authentication how to get the list of items. The error 412 may be due to parameters being out of order.

gabovos commented 4 years ago

hi @ninjageek-zcs ,can you upload your k8s_collect.pl file after modifying the JSON :: RPC :: Legacy :: Client

gabovos commented 4 years ago

hello @MogiePete , I have the same problem as you, I run auth.pl and it works for me but k8s_collect.pl still fails me with "Zabbix API HTTP Connection Error at ./k8s_collect.pl line 66.", did you get to solve it?

b2ronn commented 4 years ago

$ua-> default_header('Content-Type' => 'application/json-rpc'); does not work. I added my @content_type = ( 'content-type', 'application/json', ); ....... my $respAuth = $ua->post($APIURL, @content_type, Content => $authRequestJson);

and in each POST