ruby / net-http

Net::HTTP provides a rich library which can be used to build HTTP user-agents.
Other
98 stars 66 forks source link

Generation of multipart post fail if inconsistent encodings given as input #187

Open yxhuvud opened 1 week ago

yxhuvud commented 1 week ago

When attaching multipart entities of different encodings it is easy to get failures due to intermediary strings inside encode_multipart_form_data that care about encoding despite there being no need to care about them.

Example:

require "net/http"
require "stringio"

utf_string = "räksmörgås"
ascii_string = "räkmacka"
ascii_string.force_encoding("ASCII-8BIT")

form_data = {
  "foo"   => "bar",
  "utf8"  => StringIO.new(utf_string),
  "ascii" => StringIO.new(ascii_string)
}

url = URI("http://www.example.com")

request = Net::HTTP::Post.new(url)
request.set_form(form_data, "multipart/form-data")

Net::HTTP.start(url.hostname, url.port) do |http|
  http.request(request)
end

This results in

net-http-0.4.1/lib/net/http/generic_request.rb:354:in `block in encode_multipart_form_data': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError)
    from /home/linus/.gem/ruby/3.3.3/gems/net-http-0.4.1/lib/net/http/generic_request.rb:320:in `each'
    from /home/linus/.gem/ruby/3.3.3/gems/net-http-0.4.1/lib/net/http/generic_request.rb:320:in `encode_multipart_form_data'
    from /home/linus/.gem/ruby/3.3.3/gems/net-http-0.4.1/lib/net/http/generic_request.rb:303:in `send_request_with_body_data'
    from /home/linus/.gem/ruby/3.3.3/gems/net-http-0.4.1/lib/net/http/generic_request.rb:204:in `exec'
[...]

There are real use cases that are reasonable like posting both a text file and a pdf at the same time.

A workaround is to always encode absolutely everything as ascii-8bit before posting it.

yxhuvud commented 5 days ago

Looking at encode_multipart_form_data, I'm guessing this would stop being an issue if the data was flushed on every key.