Provide more control over charset parameter when writing form data

[rstoyanchev]

Some servers don't support application/x-www-form-urlencoded with a charset parameter, and there isn't clear spec guidance. In light of this, we can avoid adding a charset parameter unless it deviates from the default UTF-8.

See #31742 for more details.

[hannahboat]

I'm also experiencing this issue. The API we're interacting with doesn't permit content-type 'application/x-www-form-urlencoded;charset=UTF-8' only 'application/x-www-form-urlencoded'. What's the eta for a release of this fix?

[bclozel]

This will be released with 6.2.0, to be released next November.

[hannahboat]

Ok that's quite some time away, is there any chance of being added to a patch release?

[rstoyanchev]

@hannahBoat the getMediaType method is protected since #22588, so you can override it as a workaround as shown in https://github.com/spring-projects/spring-framework/issues/31742#issuecomment-1839237948.

[hannahboat]

This workaround doesn't work for us unfortunately. We are using Spring Security Ouath2. Which calls into DefaultClientCredentialsTokenResponseClient

The FormHttpMessageConverter is set within the constructor of DefaultClientCredentialsTokenResponseClient

    public DefaultClientCredentialsTokenResponseClient() {
        RestTemplate restTemplate = new RestTemplate(Arrays.asList(new FormHttpMessageConverter(), new OAuth2AccessTokenResponseHttpMessageConverter()));
        restTemplate.setErrorHandler(new OAuth2ErrorResponseErrorHandler());
        this.restOperations = restTemplate;
    }

It's the FormHttpMessageConverter that's causing the issue. In particular the writeForm method. This append's the charset to the content-type header. There's a line that then throws an error if the charset is absent.

    private void writeForm(MultiValueMap<String, Object> formData, @Nullable MediaType contentType,
            HttpOutputMessage outputMessage) throws IOException {

        contentType = getFormContentType(contentType);
        outputMessage.getHeaders().setContentType(contentType);

        Charset charset = contentType.getCharset();
        Assert.notNull(charset, "No charset"); // should never occur

        byte[] bytes = serializeForm(formData, charset).getBytes(charset);
        outputMessage.getHeaders().setContentLength(bytes.length);

        if (outputMessage instanceof StreamingHttpOutputMessage streamingOutputMessage) {
            streamingOutputMessage.setBody(new StreamingHttpOutputMessage.Body() {
                @Override
                public void writeTo(OutputStream outputStream) throws IOException {
                    StreamUtils.copy(bytes, outputStream);
                }

                @Override
                public boolean repeatable() {
                    return true;
                }
            });
        }
        else {
            StreamUtils.copy(bytes, outputMessage.getBody());
        }
    }

We would need to extend FormHttpMessageConverter, re write a good proportion of the code to omit the 'charset'. Then feed this into a new 'RestTemplate'

RestTemplate restTemplate = new RestTemplate(Arrays.asList(new CustomFormHttpMessageConverter(), new OAuth2AccessTokenResponseHttpMessageConverter()));
restTemplate.setErrorHandler(new OAuth2ErrorResponseErrorHandler());

To subsequently get handle on the DefaultClientCredentialsTokenResponseClient from the Spring OAuath setup and run setRestOperations with the new restTemplate.

This feels very convoluted. Particularly when we need to security patch and update spring-web. We may need to update the 'custom' code each time. There's a risk this is not compatible.

Is there a more streamlined solution to this issue?