search

edgurgel / httpoison

Yet Another HTTP client for Elixir powered by hackney
https://hex.pm/packages/httpoison
MIT License
2.22k stars 339 forks source link

Multipart content based post failing if the file name contains space or () characters #486

Open muraleepadma opened 5 months ago

muraleepadma commented 5 months ago

For a file name ZAF (2).zip the code fails to upload it whereas if I remove the spaces and () from the file name it works.

One difference I notice is that content length and content type are different just because the file name has spaces.

{"content-length", "92"}, {"content-type", "application/json; charset=utf-8"},

Appreciate if you could look into this and let me know if any additional details needed.

Environment :

erlang 26.0.2 elixir 1.15.6-otp-26 {:httpoison, "~> 2.1"}

Error Case :

options = [connect_timeout: 3_600_000, recv_timeout: 3_600_000, timeout: 3_600_000]
path_to_file = "/Users/mpadmaos10/Downloads/ZAF\ \(2\).zip"
content = File.read!(path_to_file)
file_name = "ZAF (2).zip"
url =  "http://localhost:52222/api/upsert_activity_attachment/user_id/mpadma/activity_id/450"
headers=[{"content-type", MIME.from_path(path_to_file)}]

HTTPoison.post(url,  {:multipart,[{"attachment", content, {"form-data", [name: "attachment", filename: file_name]}, []}]},  headers,  options)

Response : 

{:ok,
 %HTTPoison.Response{
   status_code: 400,
   body: "# Plug.Parsers.BadEncodingError at POST /api/upsert_activity_attachment/user_id/mpadma/acivity_id/450\n\nException:\n\n    ** (Plug.Parsers.BadEncodingError) invalid UTF-8 on multipart body, got byte 205\n        (plug 1.14.2) lib/plug/conn/utils.ex:292: Plug.Conn.Utils.do_validate_utf8!/3\n        (plug 1.14.2) lib/plug/parsers/multipart.ex:206: Plug.Parsers.MULTIPART.parse_multipart_headers/5\n        (plug 1.14.2) lib/plug/parsers/multipart.ex:186: Plug.Parsers.MULTIPART.parse_multipart/5\n        (plug 1.14.2) lib/plug/parsers/multipart.ex:175: Plug.Parsers.MULTIPART.parse_multipart/2\n        (plug 1.14.2) lib/plug/parsers/multipart.ex:127: Plug.Parsers.MULTIPART.parse/5\n        (plug 1.14.2) lib/plug/parsers.ex:340: Plug.Parsers.reduce/8\n        (dharma 1.0.0) lib/dharma_web/endpoint.ex:1: DharmaWeb.Endpoint.plug_builder_call/2\n        (dharma 1.0.0) deps/plug/lib/plug/debugger.ex:136: DharmaWeb.Endpoint.\"call (overridable 3)\"/2\n        (dharma 1.0.0) lib/dharma_web/endpoint.ex:1: DharmaWeb.Endpoint.call/2\n        (phoenix 1.6.16) lib/phoenix/endpoint/cowboy2_handler.ex:54: Phoenix.Endpoint.Cowboy2Handler.init/4\n        (cowboy 2.10.0) /Users/mpadmaos10/projects/source_code/dharma/deps/cowboy/src/cowboy_handler.erl:37: :cowboy_handler.execute/2\n        (cowboy 2.10.0) /Users/mpadmaos10/projects/source_code/dharma/deps/cowboy/src/cowboy_stream_h.erl:306: :cowboy_stream_h.execute/3\n        (cowboy 2.10.0) /Users/mpadmaos10/projects/source_code/dharma/deps/cowboy/src/cowboy_stream_h.erl:295: :cowboy_stream_h.request_process/3\n        (stdlib 5.0.2) proc_lib.erl:241: :proc_lib.init_p_do_apply/3\n    \n\nCode:\n\n`lib/plug/conn/utils.ex`\n\n    287     defp do_validate_utf8!(&lt;&lt;_::utf8, rest::bits&gt;&gt;, exception, context) do\n    288       do_validate_utf8!(rest, exception, context)\n    289     end\n    290   \n    291     defp do_validate_utf8!(&lt;&lt;byte, _::bits&gt;&gt;, exception, context) do\n    292>      raise exception, &quot;invalid UTF-8 on \#{context}, got byte \#{byte}&quot;\n    293     end\n    294   \n    295     defp do_validate_utf8!(&lt;&lt;&gt;&gt;, _exception, _context) do\n    296       :ok\n    297     end\n    \n`lib/plug/parsers/multipart.ex`\n\n    201         {:binary, name} -&gt;\n    202           {:ok, limit, body, conn} =\n    203             parse_multipart_body(Plug.Conn.read_part_body(conn, opts), limit, opts, &quot;&quot;)\n    204   \n    205           if Keyword.get(opts, :validate_utf8, true) do\n    206>            Plug.Conn.Utils.validate_utf8!(body, Plug.Parsers.BadEncodingError, &quot;multipart body&quot;)\n    207           end\n    208   \n    209           {conn, limit, [{name, headers, body} | acc]}\n    210   \n    211         {:file, name, path, %Plug.Upload{} = uploaded} -&gt;\n    \n`lib/plug/parsers/multipart.ex`\n\n    181         {:error, :too_large, conn}\n    182       end\n    183     end\n    184   \n    185     defp parse_multipart({:ok, headers, conn}, limit, opts, headers_opts, acc) when limit &gt;= 0 do\n    186>      {conn, limit, acc} = parse_multipart_headers(headers, conn, limit, opts, acc)\n    187       read_result = Plug.Conn.read_part_headers(conn, headers_opts)\n    188       parse_multipart(read_result, limit, opts, headers_opts, acc)\n    189     end\n    190   \n    191     defp parse_multipart({:ok, _headers, conn}, limit, _opts, _headers_opts, acc) do\n    \n`lib/plug/parsers/multipart.ex`\n\n    170       parse_multipart(conn, {m2p, limit, header_opts, opts})\n    171     end\n    172   \n    173     defp parse_multipart(conn, {m2p, limit, headers_opts, opts}) do\n    174       read_result = Plug.Conn.read_part_headers(conn, headers_opts)\n    175>      {:ok, limit, acc, conn} = parse_multipart(read_result, limit, opts, headers_opts, [])\n    176   \n    177       if limit &gt; 0 do\n    178         {mod, fun, args} = m2p\n    179         apply(mod, fun, [acc, conn | args])\n    180       else\n    \n`lib/plug/parsers/multipart.ex`\n\n    122   \n    123     @impl true\n    124     def parse(conn, &quot;multipart&quot;, subtype, _headers, opts_tuple)\n    125         when subtype in [&quot;form-data&quot;, &quot" <> ...,
   headers: [
     {"cache-control", "max-age=0, private, must-revalidate"},
     {"content-length", "8895"},
     {"content-type", "text/markdown; charset=utf-8"},
     {"date", "Tue, 13 Feb 2024 18:29:59 GMT"},
     {"server", "Cowboy"}
   ],
   request_url: "http://localhost:52222/api/upsert_activity_attachment/user_id/mpadma/acivity_id/450",
   request: %HTTPoison.Request{
     method: :post,
     url: "http://localhost:52222/api/upsert_activity_attachment/user_id/mpadma/acivity_id/450",
     headers: [],
     body: {:multipart,
      [
        {"attachment",
         <<80, 75, 3, 4, 20, 0, 0, 0, 8, 0, 205, 29, 191, 86, 113, 102, 163, 15,
           81, 6, 0, 0, 58, 10, 0, 0, 12, 0, 0, 0, 67, 117, 114, 114, ...>>,
         {"form-data", [name: "attachment", filename: "ZAF (2).zip"]}, []}
      ]},
     params: %{},
     options: [
       connect_timeout: 3600000,
       recv_timeout: 3600000,
       timeout: 3600000
     ]
   }
 }}

Same File without spaces.

options = [connect_timeout: 3_600_000, recv_timeout: 3_600_000, timeout: 3_600_000]
path_to_file = "/Users/mpadmaos10/Downloads/ZAF.zip"
file_name  = "ZAF.zip"
content = File.read!(path_to_file)
headers=[{"content-type", MIME.from_path(path_to_file)}]

HTTPoison.post(url,  {:multipart,[{"attachment", content, {"form-data", [name: "attachment", filename: file_name]}, []}]},  headers,  options)

Response :

{:ok,
 %HTTPoison.Response{
   status_code: 200,
   body: "{\"id\":100,\"status\":\"ok\",\"description\":\"Activity Attachment \\\"ZAF.zip\\\" added For Id = 100.\"}",
   headers: [
     {"access-control-allow-credentials", "true"},
     {"access-control-allow-origin", "*"},
     {"access-control-expose-headers", ""},
     {"cache-control", "max-age=0, private, must-revalidate"},
     {"content-length", "92"},
     {"content-type", "application/json; charset=utf-8"},
     {"date", "Tue, 13 Feb 2024 18:50:30 GMT"},
     {"server", "Cowboy"},
     {"x-request-id", "F7OAumZclgeU2NgAAADB"}
   ],
   request_url: "http://localhost:52222/api/upsert_activity_attachment/user_id/mpadma/activity_id/450",
   request: %HTTPoison.Request{
     method: :post,
     url: "http://localhost:52222/api/upsert_activity_attachment/user_id/mpadma/activity_id/450",
     headers: [],
     body: {:multipart,
      [
        {"attachment",
         <<80, 75, 3, 4, 20, 0, 0, 0, 8, 0, 205, 29, 191, 86, 113, 102, 163, 15,
           81, 6, 0, 0, 58, 10, 0, 0, 12, 0, 0, 0, 67, 117, 114, 114, ...>>,
         {"form-data", [name: "attachment", filename: "ZAF.zip"]}, []}
      ]},
     params: %{},
     options: [
       connect_timeout: 3600000,
       recv_timeout: 3600000,
       timeout: 3600000
     ]
   }
 }}
muraleepadma commented 5 months ago

@edgurgel if you could look into this and let me know in case any additional details are required.