HTTP Post data not encoded properly under 1.9.3

[chrisburnor]

I believe I've found a bug in twurl running under ruby 1.9.3 regarding content type encoding. Specifically, I've found that reqeusts with an asterisk ('*') in the HTTP body fail under 1.9.3 but succeed under 1.8.7. I'm using the following command to test:

$ ruby --version && ./twurl --version && ./twurl /1.1/statuses/update\.json -d "status=An Introduction to the A* algorithm http://www.redblobgames.com/pathfinding/a-star/introduction.html"  -t

Under 1.8.7, I get:

ruby 1.8.7 (2011-02-18 patchlevel 334) [i686-darwin13.1.0], MBARI 0x6770, Ruby Enterprise Edition 2011.03
0.9.2
opening connection to api.twitter.com...
opened
<- "POST /1.1/statuses/update.json HTTP/1.1\r\nAccept: */*\r\nConnection: close\r\nUser-Agent: OAuth gem v0.4.7\r\nContent-Type: application/x-www-form-urlencoded\r\nAuthorization: OAuth oauth_consumer_key=\"###\", oauth_nonce=\"###
\", oauth_signature=\"###\", oauth_signature_method=\"HMAC-SHA1\", oauth_timestamp=\"###\", oauth_token=\"###\", oauth_version=\"1.0\"\r\nContent-Length: 147\r\nHost: api.twi
tter.com\r\n\r\n"
<- "status=An%20Introduction%20to%20the%20A%2a%20algorithm%20http%3a%2f%2fwww.redblobgames.com%2fpathfinding%2fa-star%2fintroduction.html"
-> "HTTP/1.1 200 OK\r\n"
... [ rest of http response ]

Under 1.9.3, I get:

ruby 1.9.3p448 (2013-06-27 revision 41675) [x86_64-darwin13.1.0]
0.9.2
opening connection to api.twitter.com...
opened
<- "POST /1.1/statuses/update.json HTTP/1.1\r\nAccept: */*\r\nUser-Agent: OAuth gem v0.4.7\r\nContent-Type: application/x-www-form-urlencoded\r\nAuthorization: OAuth oauth_consumer_key=\"###\", oauth_nonce=\"###\", oauth_signature=\"###\", oauth_signature_method=\"HMAC-SHA1\", oauth_timestamp=\"###\", oauth_token=\"###\", oauth_version=\"1.0\"\r\nConnection: close\r\nHost: api.twitter.com\r\nContent-Length: 133\r\n\r\n"
<- "status=An+Introduction+to+the+A*+algorithm+http%3A%2F%2Fwww.redblobgames.com%2Fpathfinding%2Fa-star%2Fintroduction.html"
-> "HTTP/1.1 401 Unauthorized\r\n"
... [ rest of http response ]

Note that in 1.9.3, the status parameter in the HTTP body is not actually form-urlencoded even though that's the content-type, whereas, in 1.8.7, it is.

[ryangreenberg]

Can you modify your script to also output twurl --version to ensure that the same version is being used with each version of Ruby?

[jaakko-sf]

Should the user-agent of twurl calls be "twurl 0.9.3" rather than the oauth gem agent?

[chrisburnor]

Updated script and output as per @ryangreenberg 's request.

I'm not sure what the User-Agent should be @jaakko-sf though it seems unchanged in the different verisons of Ruby.

[goatface]

Hit this bug today. Any plan to resolve it? For now, I'm stripping the asterisk character out of my automated tweets...

[ryangreenberg]

Ruby has ended support for 1.9.3 so I think the relevant question now is whether this problem exists under Ruby 2.0 and Ruby 2.1.

[andypiper]

Closing for age / inability to repro - please open a new issue if this still occurs with 0.9.3 (and Ruby 2+)