search

Cacti / plugin_webseer

Cacti Web Services Monitoring Plugin
GNU General Public License v2.0
8 stars 9 forks source link

Some sites require compression to properly redirect #18

Closed jpobeda closed 5 years ago

jpobeda commented 7 years ago

Hi there,

I've got like 20 other urls being checked the same way and they are all working fine so far.

I've set this server to check the url, and search for 200 as the other ones.

php -q webseer_process.php --id=28 --debug DEBUG: Service Check Number 0 DEBUG: Checking Cookie File DEBUG: Executing Get Request for URL:http://myserverilo.xxx.yy.zz, IP: DEBUG: cURL errno: 22 DEBUG: cURL error: The requested URL returned error: 404 Not Found DEBUG: Service Check Number 1 DEBUG: Checking Cookie File DEBUG: Executing Get Request for URL:http://myserverilo.xxx.yy.zz, IP: DEBUG: cURL errno: 22 DEBUG: cURL error: The requested URL returned error: 404 Not Found DEBUG: Service Check Number 2 DEBUG: Checking Cookie File DEBUG: Executing Get Request for URL:http://myserverilo.xxx.yy.zz, IP: DEBUG: cURL errno: 22 DEBUG: cURL error: The requested URL returned error: 404 Not Found DEBUG: pi:300, t:1503024636 (2017-08-18 14:50:36), lc:1503024216 (2017-08-18 14:43:36), ts:1, tf:3 DEBUG: failures:2, triggered:0 DEBUG: Checking for trigger DEBUG: Time to send email to admins DEBUG: Updating Statistics

BUT if run cURL from shell

curl http://myserverilo.xxx.yy.zz <!DOCTYPE html>

Moved Temporarily

Moved Temporarily

The document has moved here.

So I changed search string to 302 and the result was the same, 404.

I also tried https with the same result, in fact that was the first try.

I didn't bother to run a tcpdump because cURL is reporting right...

I thought, maybe the settings are not getting saved to the database...but

+----+-----------------+--------------+---------------+----------------+---------------+----------------+-------+
| 28 | on      | http | myserverilo | http://myserverilo.xxx.yy.zz |    | 302    |              |               |              |           |                 |              |      0 |           3 |               4 |        2 |         1 | 2017-08-18 15:07:42 | HTTP ERROR: The requested URL returned error: 404 Not Found |       404 |   0.774163 |        0.004128 |     0.005371 |      0.181001 |              0 |             0 |              1 |       |
+----+---------+------+--------------+-------------------------------+----+--------+--------------+---------------+--------------+-----------+-----------------+------------
cigamit commented 6 years ago

I thought this was resolved already. Can you confirm.

jpobeda commented 5 years ago

It seems to still be the case. Downloaded develop branch, same thing. Running poller with --debug doesn't really give you much to look at.

netniV commented 5 years ago

Do you have a proxy setup?

netniV commented 5 years ago

Also, how many redirects are occurring? There is a maximum of 4 redirects allowed:

curl_setopt($process, CURLOPT_RETURNTRANSFER, true);
curl_setopt($process, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($process, CURLOPT_MAXREDIRS, 4);
netniV commented 5 years ago

Also, do you have checkcert enabled? IF this is redirecting to a standard iLO HTTPS setup, that will not have a valid cert.

netniV commented 5 years ago

But basically, because of the error, it's never hitting the search string portion.

jpobeda commented 5 years ago

No proxy, I'd say 1 redirect but I'll check. Check cert has always been disabled.

netniV commented 5 years ago

I made a minor change, pull the latest develop version and see if that helps. Doubtful but you never know.

jpobeda commented 5 years ago

Same thing, I'm not sure..but...looking at what curl reports when pulling the URL I'm thinking that webseer might not be properly handling the redirection

Case 1 (HTTP)

curl http://mydomain.com
<!DOCTYPE html>
<html><head><title>Moved Temporarily</title></head>
<body><h1>Moved Temporarily</h1>
<p>The document has moved <a href="http://mydomain.com/start.html">here</a>.</p></body></html>

Now if you try to curl that http://mydomain.com/start.html URL, you get 404

curl --insecure http://mydomain.com/start.html
<!DOCTYPE html>
<head>
    <title>Not Found</title>
    <link rel="shortcut icon" href="data:image/x-icon;," type="image/x-icon">
</head>
<body>
<h2>Access Error: 404 -- Not Found</h2>
<pre></pre>
</body>
</html>

Case 2 (HTTPS)

curl --insecure https://mydomain.com
<!DOCTYPE html>
<html><head><title>Moved Temporarily</title></head>
<body><h1>Moved Temporarily</h1>
<p>The document has moved <a href="https://mydomain.com/start.html">here</a>.</p></body></html>

Again, if you try to curl https://mydomain.com/start.html URL, you get 404

curl --insecure https://mydomain.com/start.html
<!DOCTYPE html>
<head>
    <title>Not Found</title>
    <link rel="shortcut icon" href="data:image/x-icon;," type="image/x-icon">
</head>
<body>
<h2>Access Error: 404 -- Not Found</h2>
<pre></pre>
</body>
</html>

On webseer console

image

Does this make sense? I manually patched the code with your latest commit and no difference.

netniV commented 5 years ago

This will be something to do with the curl options being set. I'll have to look tomorrow.

netniV commented 5 years ago

Actually, I don't think it's going to be a curl option. IF you have specified a specific URL including a filename, then it's up to the web server being called to properly identify that the resource has been moved.

In this case, it does not and merely says that start.html does not exist. That means that the web server is improperly configured. With IIS, you can set options to say whether to include any resource and I'm sure apache will have the same kind of configuration.

There is nothing that curl can do about that.

jpobeda commented 5 years ago

Dude, it's a DELL idrac, any new and old browser can handle it... Go ahead and leave it closed anyway. That won't change the fact that webseer doesn't handle the redirect properly and will eventually be raised again.

netniV commented 5 years ago

Wrong. It’s not being redirected it’s being 404d

cigamit commented 5 years ago

I have tested it on my Drac. Webseer is following the redirect (302) and getting the 404 from start.html. For the check, you have to enable "requires auth" also because its returning an error, it won't return any data otherwise. Here is the full debug / verbose output. I am wondering if we should expose the ability to not follow redirects and/or alert on status codes vs text. Even though it is returning 404 with some text, we are returning a failed check because the http code itself is an error. Even a 302 is considered an error currently if we were to not follow redirects.

2019/02/17 19:35:41 - WEBSEER DEBUG: [Url 1] Executing Get Request for URL:https://X.X.X.X, IP:
* About to connect() to X.X.X.X port 443 (#2)
*   Trying X.X.X.X...
* Connected to X.X.X.X (X.X.X.X) port 443 (#2)
* skipping SSL peer certificate verification
* SSL connection using TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
*       subject: E=support@dell.com,CN=idrac-XXXXXXX,OU=Remote Access Group,O=Dell Inc.,L=Round Rock,ST=Texas,C=US
*       start date: Jan 15 21:54:00 2017 GMT
*       expire date: Jan 16 21:55:15 2027 GMT
*       common name: idrac-4N47XV1
*       issuer: E=support@dell.com,CN=idrac-XXXXXXX,OU=Remote Access Group,O=Dell Inc.,L=Round Rock,ST=Texas,C=US
> GET / HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)
Host: X.X.X.X
Accept: */*

< HTTP/1.1 302 Moved Temporarily
< Keep-Alive: timeout=60, max=199
< Content-Type: application/x-gzip
< Location: https://X.X.X.X/start.html
< Date: Mon, 18 Feb 2019 07:35:43 GMT
< ETag: Mon Feb 18 01:35:43 2019 CST
< Content-Length: 202
< Last-Modified: Sat, 05 Nov 2016 03:58:11 GMT
< Connection: Keep-Alive
< Accept-Ranges: bytes
<
* Ignoring the response-body
* Connection #2 to host X.X.X.X left intact
* Issue another request to this URL: 'https://X.X.X.X/start.html'
* Found bundle for host X.X.X.X: 0x564787a1ee60
* Re-using existing connection! (#2) with host X.X.X.X
* Connected to X.X.X.X (X.X.X.X) port 443 (#2)
> GET /start.html HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)
Host: X.X.X.X
Accept: */*

< HTTP/1.1 404 Not Found
< Keep-Alive: timeout=60, max=198
< Content-Type: text/html
< Date: Mon, 18 Feb 2019 07:35:43 GMT
< Cache-Control: no-cache
< Content-Length: 223
< Connection: Keep-Alive
< Accept-Ranges: bytes
<
* Connection #2 to host X.X.X.X left intact
HTTP/1.1 302 Moved Temporarily
Keep-Alive: timeout=60, max=199
Content-Type: application/x-gzip
Location: https://X.X.X.X/start.html
Date: Mon, 18 Feb 2019 07:35:43 GMT
ETag: Mon Feb 18 01:35:43 2019 CST
Content-Length: 202
Last-Modified: Sat, 05 Nov 2016 03:58:11 GMT
Connection: Keep-Alive
Accept-Ranges: bytes

HTTP/1.1 404 Not Found
Keep-Alive: timeout=60, max=198
Content-Type: text/html
Date: Mon, 18 Feb 2019 07:35:43 GMT
Cache-Control: no-cache
Content-Length: 223
Connection: Keep-Alive
Accept-Ranges: bytes

<!DOCTYPE html>
<head>
    <title>Not Found</title>
    <link rel="shortcut icon" href="data:image/x-icon;," type="image/x-icon">
</head>
<body>
<h2>Access Error: 404 -- Not Found</h2>
<pre></pre>
</body>
</html>
cigamit commented 5 years ago

Enabling compress by uncommenting this line

        curl_setopt($process, CURLOPT_ENCODING , $this->compression);

Will get rid of the 404 and display the full start.html page html. I will have to think about how best to fix this with out breaking sites that don't allow compression.

netniV commented 5 years ago

So the DRAC is giving out a different page depending on whether compression is on? That's whacky 😱

cigamit commented 5 years ago

Yep. I believe we can turn it on for every site, as it is just telling it that it accepts gzip compression but doesn't have to use it.

netniV commented 5 years ago

Well, I have seen some devices fail to respond because compression is on. We may have to make it optional, with the default on and see how that goes. If we find that it's causing a lot of issues, remove the default so it's defaulted to off.