GoogleCloudPlatform / webapp2

webapp2 is a framework for Google App Engine
https://webapp2.readthedocs.org
Other
142 stars 64 forks source link

Response.write() tries to decode bytes from binary stream #146

Open Mark-Hetherington opened 5 years ago

Mark-Hetherington commented 5 years ago

I'm trying to stream a file to the web browser, and to do this I'm using handler.request.write(bytes). However webapp tries to convert the bytes to text, which fails as the bytes are not valid for any character set:


            text = text.decode(self.default_charset)```
This results in an error:
`UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc8 in position 6: invalid continuation byte`

In a python2 environment writing bytes was supported.

I've tried modifying the Response.write() function to do no processing if bytes are passed in, and this allows me to stream a .gif to the browser successfully. 
I'm running in to issues with a gzip encoded SVG, although this is more an issue of application logic setting the headers correctly (or decompressing the SVG before streaming to the browser). Once I've been able to test this I think I can craft a PR to resolve this issue.
jeremydw commented 4 years ago

FYI: I ran into this issue as well, and was able to work around it by bypassing the subclass's Response.write method and called directly into the base class's write method.

The error happens here:
https://github.com/GoogleCloudPlatform/webapp2/blob/master/webapp2.py#L420

It occurs when trying to pass bytes content (e.g. a binary asset like a JPG file) as the text parameter.

I worked around it by replacing the self.response.out.write(...) code in my handler with the following, where I bypass the subclass's write method and just call into the webob.Response.write directly.

fp = open(path, 'rb')  # Path is the path to the binary file I'm trying to send.
super(webapp2.Response, self.response).write(fp.read())

FYI: This seems to work for me, but I'm not sure if there are any side effects that may crop up from this approach.