Errors dues to extra charset parameter UTF-8 in Content-Type header

[subukris]

We are using requests oauthlib Backend Application Flow (Client Credentials Grant) with a service and get an error.

{
  "error": "invalid_request",
  "error_description": "Content-Type must be set to application/x-www-form-urlencoded"
}

In the request, we see Content-Type: application/x-www-form-urlencoded;charset=UTF-8 The charset is additionally specified.

Upon investigating into the OAuth2.0 RFC https://tools.ietf.org/html/rfc6749#page-40

section 4.4.2 says:

"The client makes a request to the token endpoint by adding the following parameters using the "application/x-www-form-urlencoded" format per Appendix B with a character encoding of UTF-8 in the HTTP request entity-body"

Example:

POST /token HTTP/1.1
     Host: server.example.com
     Authorization: Basic czZCaGRSa3F0MzpnWDFmQmF0M2JW
     Content-Type: application/x-www-form-urlencoded

     grant_type=client_credentials

The spec mandates UTF-8 character encoding of the entity body. The Content-Type is set to "application/x-www-form-urlencoded" without any charset specified because that is redundant/implied information.

In the spec for application/x-www-form-urlencoded https://url.spec.whatwg.org/#application/x-www-form-urlencoded There is no charset parameter defined for this media type.

"Such logic is not described here as only UTF-8 is conforming."

There are also examples of other oauth clients which have removed the extra charset parameter from their implementation.

https://github.com/node-fetch/node-fetch/issues/576

https://github.com/request/request/issues/701

Question 1) Is it possible to override the default value of Content-Type and set it to "application/x-www-form-urlencoded" without the charset parameter to allow for a strict interpretation of the spec such as this service?

Question 2) Should the explicit charset be removed all together/made the default to make it more aligned with the rfc?

[metadaddy]

I encountered precisely this problem when using requests oauthlib and initially interpreted it as an issue in the authorization server. However, on reading the relevant specs (in particular https://url.spec.whatwg.org/#application/x-www-form-urlencoded), application/x-www-form-urlencoded should never have a charset parameter, since the spec defines a specific parsing of form data, UTF-8 with a tweak to decode "+" as " ", which isn't strictly UTF-8.

[JonathanHuot]

Fixed in https://github.com/requests/requests-oauthlib/pull/438