Cannot add nagios tac.cgi page as asset

[compuguy]

I'm trying to add a nagios Tactical Monitoring Overview page (I've gotten everything else working based on the guide here: https://www.screenly.io/use-cases/dashboard/nagios/, but the python requests library doesn't like the page. Running the url in the python shell returns this (as done in past issues, including #494):

Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 97, in head return request('head', url, kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 57, in request return session.request(method=method, url=url, kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 475, in request resp = self.send(prep, send_kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 585, in send r = adapter.send(request, kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 453, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",))

Output from daemon.log:

Dec 6 17:01:53 raspberrypi python[631]: raise Exception("Could not retrieve file. Check the asset URL.") Dec 6 17:01:53 raspberrypi python[631]: Exception: Could not retrieve file. Check the asset URL. Dec 6 17:01:54 raspberrypi python[631]: /usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for blah has no subjectAltName, falling back to check for a commonName for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.) Dec 6 17:01:54 raspberrypi python[631]: SubjectAltNameWarning Dec 6 17:01:54 raspberrypi python[631]: Traceback (most recent call last): Dec 6 17:01:54 raspberrypi python[631]: File "/home/pi/screenly/server.py", line 213, in api_view Dec 6 17:01:54 raspberrypi python[631]: return make_json_response(view(*args, **kwargs)) Dec 6 17:01:54 raspberrypi python[631]: File "/home/pi/screenly/server.py", line 227, in add_asset Dec 6 17:01:54 raspberrypi python[631]: raise Exception("Could not retrieve file. Check the asset URL.") Dec 6 17:01:54 raspberrypi python[631]: Exception: Could not retrieve file. Check the asset URL.

[vpetersson]

At a quick glance, my first gut feeling would be that it has to do with the SSL certificate. Are you using a self-signed cert?

[compuguy]

Yes, but I installed it on the box, plus I maintain a fork of certifi, that looks at the concatenated ca-certificates.crt that debian based distros use: https://github.com/compuguy/python-certifi/tree/ubuntu

Edit: I should also mention the /nagios3/ homepage/frame loads without issue and can be added to screenly. I've also disabled cert verification, which still results in an error.

[vpetersson]

It's interesting that you say that it works with certain pages on the same server, and not others. The relevant error appears to be requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",)).

I did some research and found that this issue that mentions the same error. They mention that it is related to httplib and most likely because of some error in the upstream server.

First, I'd suggest that you try upgrading requests to the latest version (pip install -U requests). If that doesn't help, try the following:

$ curl -I https://your/server/path

(I'm curious about the HTTP status code).

[compuguy]

Sure thing (Note that parts of the output will be censored):

curl -I https://*Host*/special/cgi-bin/tac.cgi HTTP/1.1 200 OK Date: Thu, 08 Dec 2016 17:48:11 GMT Server: Apache/2.4.7 (Ubuntu) Cache-Control: no-store Pragma: no-cache Refresh: 90 Expires: Thu, 01 Jan 1970 00:00:00 GMT Last-Modified: Thu, 08 Dec 2016 17:48:11 GMT Vary: Accept-Encoding Content-Type: text/html;charset=utf-8

Also updated the requests library, no dice:

Dec 8 17:50:14 raspberrypi python[609]: /usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/connection.py:337: SubjectAltNameWarning: Certificate for blah has no subjectAltName, falling back to check for a commonName for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.) Dec 8 17:50:14 raspberrypi python[609]: SubjectAltNameWarning Dec 8 17:50:14 raspberrypi python[609]: Traceback (most recent call last): Dec 8 17:50:14 raspberrypi python[609]: File "/home/pi/screenly/server.py", line 213, in api_view Dec 8 17:50:14 raspberrypi python[609]: return make_json_response(view(*args, **kwargs)) Dec 8 17:50:14 raspberrypi python[609]: File "/home/pi/screenly/server.py", line 227, in add_asset Dec 8 17:50:14 raspberrypi python[609]: raise Exception("Could not retrieve file. Check the asset URL.") Dec 8 17:50:14 raspberrypi python[609]: Exception: Could not retrieve file. Check the asset URL.

[vpetersson]

Very interesting. So curl does get a 200 HTTP Status somewhere in the Python Requests stack, something chokes.

What version of Nagios are you running? Maybe there's a known bug.

[compuguy]

Nagios Core 3.5.1. I'll look into if there is a nagios bug. I've not seen issues with piSignage or other browsers (the latest version uses chromium vs uzbl).

[vpetersson]

Thanks, yes, it's most likely related to something inside of the Python/Requests stack. We'll see if we're able to reproduce this using that version of Nagios.

[jonwitts]

Hi there,

We are running into this same issue. I can reproduce the same errors on both a https and http connection so I think that rules out the certificate being at play... We are running Nagios Core 4.1.1

The daemon.log file shows this when trying to add the nagios URL:

Sep 19 11:10:01 raspberrypi python[546]: Traceback (most recent call last): Sep 19 11:10:01 raspberrypi python[546]: File "/home/pi/screenly/server.py", line 202, in api_view Sep 19 11:10:01 raspberrypi python[546]: return make_json_response(view(*args, **kwargs)) Sep 19 11:10:01 raspberrypi python[546]: File "/home/pi/screenly/server.py", line 238, in add_asset Sep 19 11:10:01 raspberrypi python[546]: raise Exception("Could not retrieve file. Check the asset URL.") Sep 19 11:10:01 raspberrypi python[546]: Exception: Could not retrieve file. Check the asset URL. Sep 19 11:10:07 raspberrypi python[732]: Last update: 2017-09-19 11:02:15.237273

and when trying to access the URL from Python Requests we get the following:

pi@raspberrypi:~ $ python Python 2.7.13 (default, Jan 19 2017, 14:48:08) [GCC 6.3.0 20170124] on linux2 Type "help", "copyright", "credits" or "license" for more information.

import requests r = requests.head('http://nagios.localnetwork/abc123/cgi-bin/tac.cgi') Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 97, in head return request('head', url, kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 57, in request return session.request(method=method, url=url, kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 475, in request resp = self.send(prep, send_kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 585, in send r = adapter.send(request, kwargs) File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 453, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",)) exit()

[vpetersson]

@jonwitts interesting find. So it seems like Nagios won't return an appropriate HEAD status. What do you get if you do a request.get()?

[jonwitts]

A request.get returns the following...

import requests r = requests.get('http://nagios.localnetwork/abc123/cgi-bin/tac.cgi') print r.status_code 200 print r <Response [200]>

[vpetersson]

Cool. So the bug is that it doesn't handle HEAD requests properly.

[vpetersson]

The quick solution would to stick something like Nginx in front of Nagios and reverse proxy. The longer solution is to patch Screenly OSE such that the HEAD response is optional.

[jonwitts]

Or the even quicker (and probably a whole lot less stable!) solution is to comment out lines 183 - 190 from ~/screenly/lib/utils.py ;-)

    #if requests.head(
    #    url,
    #    allow_redirects=True,
    #    headers=headers,
    #    timeout=10,
    #    verify=verify
    #).status_code in HTTP_OK:
    #    return False

[vpetersson]

Yeah, that's another solution, but you'd have to manually patch with every upgrade.

[jonwitts]

Yeah I appreciate that. It's only the one screen in our IT Support office which needs to load the Nagios page so I am not too fussed about us breaking that one on an upgrade :-D

Would it be possible to change the utils.py check so that it had to fail both head and get checks before it returned false?

I am not too clear about what would cause a failure in the requests.head...

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.