search

fireantology / django-logtailer

Allows the viewing of any log file entries in real time directly from the Django admin interface. It allows you to filter on logs with regex and offer also a log clipboard for save desired log lines to the django db.
BSD 3-Clause "New" or "Revised" License
97 stars 30 forks source link

UnicodeDecodeError raised by log view #15

Open kmierzeje opened 4 years ago

kmierzeje commented 4 years ago

UnicodeDecodeError at /admin/logs/get-log-line/1/ 'ascii' codec can't decode byte

Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, *callback_args, *callback_kwargs) File "/usr/local/lib/python3.8/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view return view_func(request, args, **kwargs) File "/usr/local/lib/python3.8/site-packages/logtailer/views.py", line 56, in get_log_lines content = get_history(file, history) File "/usr/local/lib/python3.8/site-packages/logtailer/views.py", line 31, in get_history data.append(f.read(buffer_size)) File "/usr/lib64/python3.8/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0]

The error makes it impossible to view log. Please make the encoding configurable is settings or in LogFile model or most preferably in both places.

Even if correct encoding is configured, the error might still appear if log contains invalid codepoints. logtailer should be resilient to such characters - replace them with '?' or something...

Django Version: 3.0.8 Python Version: 3.8.3

fireantology commented 4 years ago

Can you upload a log file that reproduces the error ?

Thanks Regards

kmierzeje commented 4 years ago

test.log

I reproduced it on Windows, so default codec is cp1250 not ascii, but effect is the same. I got a different call stack:

UnicodeDecodeError: 'charmap' codec can't decode byte 0x98 in position 5: character maps to

Traceback (most recent call last): File "C:\apps\Python37\lib\site-packages\django\core\handlers\exception.py", line 47, in inner response = get_response(request) File "C:\apps\Python37\lib\site-packages\django\core\handlers\base.py", line 179, in _get_response response = wrapped_callback(request, *callback_args, *callback_kwargs) File "C:\apps\Python37\lib\site-packages\django\contrib\auth\decorators.py", line 21, in _wrapped_view return view_func(request, args, **kwargs) File "C:\apps\Python37\lib\site-packages\logtailer\views.py", line 64, in get_log_lines for line in file: File "C:\apps\Python37\lib\encodings\cp1250.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0]