Requests without charset are processed incorrectly in 2.5 version

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Send POST request without content-type (for example ajax POST request from 
Google Chrome
2. You will see request headers like:
Content-Length:57
Content-Type:application/x-www-form-urlencoded

Then when you will access self.request.POST in webapp 2.5 you will have values 
as basestrings instead of unicode. The bug is introduced in
http://code.google.com/p/webapp-improved/source/detail?r=d36c461b86ba91fa9a859cf
d8280bbe03d431380#

If browser doesn't sends charset in Content-Type header, webapp2 should use 
encoding used for encoding responses ('utf-8' in most cases). ATM webapp2.5 
can't properly process form submissions sent by Chrome for example. 

Please revert d36c461b86ba91fa9a859cfd8280bbe03d431380.

Original issue reported on code.google.com by dogada on 5 Feb 2012 at 11:21

[GoogleCodeExporter]

Ok, I'll ask around what to do.

Ref: issue 23 and issue 39

Original comment by rodrigo.moraes on 5 Feb 2012 at 3:27

[GoogleCodeExporter]

Hey dogada, are you using webob before version 1.1, right (e.g. by using Python 
2.5 on App Engine)?

Because I can't see how this can occur with webob 1.1 - fetching the charset in 
webob 1.1 returns 'UTF-8' if the charset is not set:
https://github.com/Pylons/webob/blob/a9dafaefb2f1c91e7a08bf2dd95c9a5d73628e7a/we
bob/request.py#L187

I also can't see how this can occur with webob 1.2 - the charset settings 
functions are deprecated and UTF-8 is assumed.

Original comment by bquin...@google.com on 6 Feb 2012 at 5:56

[GoogleCodeExporter]

Original comment by bquin...@google.com on 6 Feb 2012 at 6:34

[GoogleCodeExporter]

I use latest 1.6.2 SDK and python27 runtime. In app.yaml I have:
 libraries:
- name: webob
  version: "1.1.1"

However as I discovered now 1.6.2 SDK bundled with WebOb 0.9 version that is in 
use even you request WebOb 1.1.1 in app.yaml. And reported in this ticket 
problem is reproduced with default (bundled with 1.6.2 SDK) WebOb.

After `pip install -I webob==1.1.1` (I selected 1.1.1 because it's installed in 
production) the problem with charset is disappeared. It even makes possible 
fully remove charset detecting workaround from webapp2 code because it 
duplicates logic in webob 1.1 (see attached patch). I tested modified webapp 
2.5 version with WebOb 1.1.1 and it works fine for me (you can see modified 
version here: http://www.cliws.com/demo/).

IMO somewhere in the docs should be stated that webapp2.5 should be used with 
WebOb 1.1 at least and will not work properly with WebOb 0.9 bundled with 
current 1.6.2 SDK.

Original comment by dogada on 6 Feb 2012 at 9:37

Attachments:

• webapp2_charset.diff

[GoogleCodeExporter]

I think I can add a workaround based on detected WebOb version... for the time 
being.

Original comment by rodrigo.moraes on 6 Feb 2012 at 11:06

[GoogleCodeExporter]

Rodrigo: does that mean that you will fix this? I was planning on submitting a 
fix sometime today.

Original comment by bquin...@google.com on 6 Feb 2012 at 7:38

[GoogleCodeExporter]

Go ahead. :) Was thinking out loud.

Original comment by rodrigo.moraes on 6 Feb 2012 at 8:18

[GoogleCodeExporter]

Fixed in 577172396b29

Original comment by bquin...@google.com on 7 Feb 2012 at 4:40

• Changed state: Fixed

[GoogleCodeExporter]

2.5.1, just released, contain Brian's fix. Thanks for the feedback!

Original comment by rodrigo.moraes on 7 Feb 2012 at 11:59