pallets-eco / flask-debugtoolbar

A toolbar overlay for debugging Flask applications
https://flask-debugtoolbar.readthedocs.io
BSD 3-Clause "New" or "Revised" License
953 stars 146 forks source link

UUID support in URLs #112

Open DenisKuplyakov opened 7 years ago

DenisKuplyakov commented 7 years ago

Thanks for the good extension that helps a lot!

I have problems using it with Flask-UUID extension and URLs that have UUIDs inside. Is it possible to fix this? Providing stack trace below:

File "..prog_path/e_venv/lib/python2.7/site-packages/flask/app.py", line 1997, in __call__
return self.wsgi_app(environ, start_response)
File "..prog_path/e_venv/lib/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
response = self.handle_exception(e)
File "..prog_path/e_venv/lib/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
reraise(exc_type, exc_value, tb)
File "..prog_path/e_venv/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "..prog_path/e_venv/lib/python2.7/site-packages/flask/app.py", line 1615, in full_dispatch_request
return self.finalize_request(rv)
File "..prog_path/e_venv/lib/python2.7/site-packages/flask/app.py", line 1632, in finalize_request
response = self.process_response(response)
File "..prog_path/e_venv/lib/python2.7/site-packages/flask/app.py", line 1856, in process_response
response = handler(response)
File "..prog_path/e_venv/lib/python2.7/site-packages/flask_debugtoolbar/__init__.py", line 223, in process_response
toolbar_html = toolbar.render_toolbar()
File "..prog_path/e_venv/lib/python2.7/site-packages/flask_debugtoolbar/toolbar.py", line 46, in render_toolbar
return template.render(**context)
File "..prog_path/e_venv/lib/python2.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "..prog_path/e_venv/lib/python2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "..prog_path/e_venv/lib/python2.7/site-packages/flask_debugtoolbar/templates/base.html", line 50, in top-level template code
{{ panel.content()|safe }}
File "..prog_path/e_venv/lib/python2.7/site-packages/flask_debugtoolbar/panels/sqlalchemy.py", line 122, in content
'signed_query': dump_query(query.statement, query.parameters),
File "..prog_path/e_venv/lib/python2.7/site-packages/flask_debugtoolbar/panels/sqlalchemy.py", line 34, in dump_query
return query_signer().dumps([statement, params])
File "..prog_path/e_venv/lib/python2.7/site-packages/itsdangerous.py", line 565, in dumps
payload = want_bytes(self.dump_payload(obj))
File "..prog_path/e_venv/lib/python2.7/site-packages/itsdangerous.py", line 847, in dump_payload
json = super(URLSafeSerializerMixin, self).dump_payload(obj)
File "..prog_path/e_venv/lib/python2.7/site-packages/itsdangerous.py", line 550, in dump_payload
return want_bytes(self.serializer.dumps(obj))
File "..prog_path/e_venv/lib/python2.7/site-packages/itsdangerous.py", line 51, in dumps
return json.dumps(obj, separators=(',', ':'))
File "/usr/lib64/python2.7/json/__init__.py", line 251, in dumps
sort_keys=sort_keys, **kw).encode(obj)
File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode
return _iterencode(o, 0)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe8 in position 0: invalid continuation byte
mgood commented 7 years ago

The stack trace is from the SQLAlchemy panel, so I think the issue is with serializing one of your SQL query parameters, not necessarily with the URL.

What parameters are you passing to the SQL query? Also, what version of the extensions are you using?

DenisKuplyakov commented 7 years ago

I have some models with UUID as their primary key. I use UUID type from SQLAlchemy-Utils package. Endpoints that uses UUID in URL have queries by UUID from the tables.

Versions of packages that may be interesting (actually there is more stuff like redis, marshmallow, etc.):

Flask==0.12.1
Flask-DebugToolbar==0.10.1
Flask-SQLAlchemy==2.2
Flask-UUID==0.2
SQLAlchemy==1.1.9
SQLAlchemy-Utils==0.32.14
DenisKuplyakov commented 7 years ago

You are right, the real problem is random UUIDs that are serialized to strings and inserted to queries. As a result such strings contain invalid utf-8 characters. Below is the example:

[
   u'SELECT client.id AS client_id, client.name AS client_name \nFROM client \nWHERE client.id = %s',
   ('s\xa7Gf\x0f\xadEL\x87\xef{$ya2t')
]

Now I'm using toolbar with sqlalchemy panel disabled. Why debug toolbar tries to dump list with query and parameters to json?