search

bdd / runitor

A command runner with healthchecks.io integration
BSD Zero Clause License
280 stars 15 forks source link

UnicodeDecodeError on success ping with special characters from a Windows host #28

Closed marinbernard-pep06 closed 3 years ago

marinbernard-pep06 commented 3 years ago

Hi,

We use Runitor to monitor scheduled tasks involving PowerShell on Windows Server boxes. Some of our scripts may output a few line of text on stdout. In that case, the success ping may fail, as Django may reject it with a UnicodeDecodeError. So far, it seems the exception is raised only when the logged output contains special (non-latin1) characters.

We're using HealthChecks 1.23 with Runitor 0.8.0. HealthChecks 1.23 fixed a bug involving Unicode characters in webhook headers. I hoped the issue would disappear after upgrading to the new version, but it unfortunately did not. I don't know which project should deal with this issue. If this is a HealthChecks error, please let me know.

Here is the transcript of a job failing to send its success ping. The lines between the Runitor invocation and the Runitor error message were printed by the script. Note the Espace disque lib?r??:0 string which contains mangled special (French) characters:

C:\Users\bernardm>"C:\Program Files\Runitor\Runitor.exe" -api-url "https://<host>/ping" -uuid "<uuid>" -- "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -Command "& { Start-WsusMaintenance }"
Declining expired updates...
Declining superseeded updates...
Deleting obsolete computers...
Deleting obsolete updates...
Performing database maintenance...
Shrinking database files...
Removing unneeded content files...
Espace disque lib?r??:0
Starting a garbage collection job on the volume storing content files...
2021/10/18 14:27:32 PingSuccess: Post "https://<host>/ping/<uuid>": net/http: HTTP/1.x transport connection broken: http: ContentLength=318 with Body length 0

Running the very same script without the Runitor wrapper preserves special characters:

C:\Users\bernardm>"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -Command "& { Start-WsusMaintenance }"
Declining expired updates...
Declining superseeded updates...
Deleting obsolete computers...
Deleting obsolete updates...
Performing database maintenance...
Shrinking database files...
Removing unneeded content files...
Espace disque libéré :0
Starting a garbage collection job on the volume storing content files...

Here is the Django exception. I removed the settings; let me know if you need them.

Internal Server Error: /ping/<uuid>

UnicodeDecodeError at /ping/<uuid>
'utf-8' codec can't decode byte 0x82 in position 237: invalid start byte

Request Method: POST
Request URL: https://<host>/ping/<uuid>
Django Version: 3.2.8
Python Executable: /opt/healthchecks/venv/bin/python3
Python Version: 3.8.10
Python Path: ['/opt/healthchecks/healthchecks', '/opt/healthchecks/venv/bin', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/opt/healthchecks/venv/lib/python3.8/site-packages']
Server time: Mon, 18 Oct 2021 12:27:31 +0000 Installed Applications:
('hc.accounts',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.humanize',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'compressor',
 'hc.api',
 'hc.front',
 'hc.payments')
Installed Middleware:
('django.middleware.security.SecurityMiddleware',
 'whitenoise.middleware.WhiteNoiseMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'hc.accounts.middleware.CustomHeaderMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'hc.accounts.middleware.TeamAccessMiddleware')

Traceback (most recent call last):
  File "/opt/healthchecks/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/opt/healthchecks/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/healthchecks/venv/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/opt/healthchecks/venv/lib/python3.8/site-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/opt/healthchecks/healthchecks/hc/api/views.py", line 50, in ping
    body = request.body.decode()

Exception Type: UnicodeDecodeError at /ping/<uuid>
Exception Value: 'utf-8' codec can't decode byte 0x82 in position 237: invalid start byte Request information:
USER: AnonymousUser

GET: No GET data

POST: No POST data

FILES: No FILES data

COOKIES: No cookie data

META:
CONTENT_LENGTH = '318'
CONTENT_TYPE = 'text/plain'
HTTP_ACCEPT_ENCODING = 'gzip'
HTTP_CONNECTION = 'close'
HTTP_HOST = '<uuid>'
HTTP_USER_AGENT = 'runitor/v0.8.0 (+https://bdd.fi/x/runitor)'
HTTP_X_FORWARDED_FOR = '10.6.185.217, 10.6.104.217'
HTTP_X_FORWARDED_PROTO = 'https'
PATH_INFO = '/ping/<uuid>'
QUERY_STRING = ''
RAW_URI = '/ping/<uuid>'
REMOTE_ADDR = '127.0.0.1'
REMOTE_PORT = '53224'
REQUEST_METHOD = 'POST'
SCRIPT_NAME = ''
SERVER_NAME = '127.0.0.1'
SERVER_PORT = '8000'
SERVER_PROTOCOL = 'HTTP/1.0'
SERVER_SOFTWARE = 'gunicorn/20.1.0'
gunicorn.socket = <socket.socket fd=10, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 8000), raddr=('127.0.0.1', 53224)> wsgi.errors = <gunicorn.http.wsgi.WSGIErrorsWrapper object at 0x7f0632eb00d0> wsgi.file_wrapper = <class 'gunicorn.http.wsgi.FileWrapper'> wsgi.input = <gunicorn.http.body.Body object at 0x7f0632ea2760> wsgi.input_terminated = True wsgi.multiprocess = False wsgi.multithread = False wsgi.run_once = False wsgi.url_scheme = 'https'
wsgi.version = '(1, 0)'

Settings:
<snip>

Many thanks for any help!

marinbernard-pep06 commented 3 years ago

FYU, I just modified the script to remove the output line which included special characters. After this change, the success ping was sent successfully.

bdd commented 3 years ago

Is this a UTF-16 Little Endian output issue with Windows?

Runitor naïvely treats stderr and stdout as just byte streams, but the backend Healtchecks running on Django seems to be expecting it to be UTF-8.

Are you asking for a feature request from Runitor, so we keep this issue open or do you want to close it?

marinbernard-pep06 commented 3 years ago

Is this a UTF-16 Little Endian output issue with Windows?

Yes it is.

Are you asking for a feature request from Runitor, so we keep this issue open or do you want to close it?

POSTing a simple string with special chars with the Invoke-RestMethod PowerShell cmdlet triggers the same error. The only way to avoid it is to explicitly build a UTF-8 string object, for use as the POST body.

Since the error may be reproduced without Runitor being involved at all, I think we'd better close the issue. I'll open another at HealthChecks.

Thank you!

marinbernard-pep06 commented 3 years ago

FYI, I opened another issue at HealthChecks' GH.