URL-encoded form data is decoded incorrectly if it contains non-ASCII text

[brianberns]

Suave seems to assume that URL-encoded data should always be decoded as ASCII, which means that non-ASCII form data is decoded incorrectly. Consider the follow program, which simply echoes the posted form's text value:

request (fun req -> Successful.OK req.["text"].Value)
    |> startWebServer defaultConfig

We can invoke this from PowerShell as follows:

(Invoke-WebRequest -Uri http://localhost:8080 -Method POST -Body @{text="test ááá"}).Content

The input is test ááá, but the output is test ááá. Explicitly setting the content type via -ContentType "text/plain; charset=utf-8" has no effect on the result.

I did a little debugging, and I think the problem is in parseData:

if d.Length = 2 then (d.[0], Some <| System.Net.WebUtility.UrlDecode(d.[1]))

Unfortunately, the .NET method WebUtility.UrlDecode always "replaces hexadecimal escape sequences with corresponding ASCII character equivalents". I think that HttpUtility.UrlDecode might be a better choice instead, since it allows an encoding to be specified.

[brianberns]

Sorry, it looks like I simply forgot to call setMimeType "text/plain; charset=utf-8" on my response. I'm double-checking now, but I think Suave is probably functioning correctly here, in which case the bug can be closed.