search

moinwiki / moin-1.9

MoinMoin Wiki (1.9, also: 1.5a ... 1.8), stable, for production wikis
https://moinmo.in/
Other
140 stars 54 forks source link

traceback in werkzeug for fallback:iso-8859-1 error handler #37

Closed GuenterBurgstaller closed 4 years ago

GuenterBurgstaller commented 6 years ago

In MoinMoin 1.9.10 all pages that have a <<Hits>> hit counter:

2018-09-14 21:40:16,034 ERROR MoinMoin.macro:139 Macro Hits (page: 'Rückblick/2018/Vorwort') raised an exception:
Traceback (most recent call last):
  File "/var/moin/1.9.10/lib/python2.7/site-packages/MoinMoin/macro/__init__.py", line 129, in execute
    return execute(self, args)
  File "/var/moin/1.9.10/lib/python2.7/site-packages/MoinMoin/macro/__init__.py", line 114, in <lambda>
    _self.request, call, _args, [_self])
  File "/var/moin/1.9.10/lib/python2.7/site-packages/MoinMoin/wikiutil.py", line 1965, in invoke_extension_function
    return function(*fixed_args, **kwargs)
  File "/var/moin/1.9.10/lib/python2.7/site-packages/MoinMoin/macro/Hits.py", line 29, in macro_Hits
    cache_days, cache_views, cache_edits = hitcounts.get_data(pagename, request, filterpage=pagename)
  File "/var/moin/1.9.10/lib/python2.7/site-packages/MoinMoin/stats/hitcounts.py", line 88, in get_data
    for event in log.reverse():
  File "/var/moin/1.9.10/lib/python2.7/site-packages/MoinMoin/logfile/__init__.py", line 116, in reverse
    result = self.previous()
  File "/var/moin/1.9.10/lib/python2.7/site-packages/MoinMoin/logfile/__init__.py", line 323, in previous
    result = self.__previous()
  File "/var/moin/1.9.10/lib/python2.7/site-packages/MoinMoin/logfile/__init__.py", line 312, in __previous
    return self.parser(self.__buffer.lines[self.__rel_index])
  File "/var/moin/1.9.10/lib/python2.7/site-packages/MoinMoin/logfile/eventlog.py", line 67, in parser
    return long(time_usecs), eventtype, wikiutil.parseQueryString(kvpairs)
  File "/var/moin/1.9.10/lib/python2.7/site-packages/MoinMoin/wikiutil.py", line 132, in parseQueryString
    decode_keys=False, include_empty=False)
  File "/var/moin/1.9.10/lib/python2.7/site-packages/MoinMoin/support/werkzeug/urls.py", line 734, in url_decode
    include_empty, errors))
  File "/var/moin/1.9.10/lib/python2.7/site-packages/MoinMoin/support/werkzeug/datastructures.py", line 409, in __init__
    for key, value in mapping or ():
  File "/var/moin/1.9.10/lib/python2.7/site-packages/MoinMoin/support/werkzeug/urls.py", line 793, in _url_decode_impl
    yield key, url_unquote_plus(value, charset, errors)
  File "/var/moin/1.9.10/lib/python2.7/site-packages/MoinMoin/support/werkzeug/urls.py", line 556, in url_unquote_plus
    return url_unquote(s, charset, errors)
  File "/var/moin/1.9.10/lib/python2.7/site-packages/MoinMoin/support/werkzeug/urls.py", line 535, in url_unquote
    rv = rv.decode(charset, errors)
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
LookupError: unknown error handler name 'fallback:iso-8859-1'

After commenting out the hit counter the pages load normally. You have the phenomenon in your own wiki. See:

https://moinmo.in/EventStats/HitCounts

It says:

<<StatsChart: Ausführung fehlgeschlagen [unknown error handler name 'fallback:iso-8859-1'] (siehe auch die Log-Datei)>>

ThomasWaldmann commented 4 years ago

page: 'Rückblick/2018/Vorwort' < from the log in the original post.

ThomasWaldmann commented 4 years ago

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=867239

ThomasWaldmann commented 4 years ago

https://werkzeug.palletsprojects.com/en/0.16.x/unicode/#error-handling

so the fallback:... handler is still a documented feature of current werkzeug.

ThomasWaldmann commented 4 years ago 
>>> werkzeug.url_unquote('%fcbel', errors='fallback:iso-8859-1')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "werkzeug/urls.py", line 610, in url_unquote
    rv = rv.decode(charset, errors)
  File "/home/user/.pyenv/versions/2.7.15/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
LookupError: unknown error handler name 'fallback:iso-8859-1'

>>> werkzeug.__version__
'0.15.6'
ThomasWaldmann commented 4 years ago

https://github.com/pallets/werkzeug/issues/1706

ThomasWaldmann commented 4 years ago

Note: I removed all comments relating to high cpu load if the event-log is rather long. That is a known problem and can be fixed by shorting / rotating it regularly.

ThomasWaldmann commented 4 years ago

There are 2 places in wikiutil that make use of fallback:....

So, if this does not get fixed in werkzeug 0.16.1, we could just wrap werkzeug.url_unquote and .url_decode and try utf-8 first and (in the exception handler) iso-8859-1 second.