JuliaWeb / HTTP.jl

HTTP for Julia
https://juliaweb.github.io/HTTP.jl/stable/
Other
626 stars 177 forks source link

add handling of multipart/mixed #1100

Open hhaensel opened 10 months ago

hhaensel commented 10 months ago

Currently, only multipart/form-data is supported by multipart handling.

While this is probably the most important use case, also multipart/mixed is quite common, e.g. in REST APIs. I recently struggled with batch requesting a SharepointOnline list and finally found my way through it. I thought, I could share my experience with the community and propose this PR.

If this seems useful to the maintainers, I'll happily provide also some test routines.

In order to add the mixed functionality I added a field type to the Form object and I added a bypass for content_disposition check, because that is not present for mixed type multiparts. Furthermore, I added parse_multipart() as a general parsing method, which automatically choses the correct parsing method from the type field. Finally I added a type-based and a handler-based parsing interface for users and a predefined parser for Responses.

Here is the intended usage of the new feature

# some preparation for SharepointOnline
using JSON3
spo_params(query) = Dict(Symbol("\$$k") => v for (k, v) in query)
spo_params(; query...) = spo_params(query)

function spo_parse(x)
    pos = findfirst(==(UInt8('{')), x)
    pos === nothing && return nothing

    d = JSON3.read(x[pos:end], Dict)["d"]
    return haskey(d, "results") ? d["results"] : d
end

# -----------

base_url = "mydemo.sharepoint.com"
token = "my-very-secret-token"

url = "$base_url/_api/web/lists"
headers = [:Accept => "application/json;odata=verbose"]

m1 = HTTP.multipart_request(:GET, url, "application/http", "binary"; headers, spo_params(top = 2)...)
m2 = HTTP.multipart_request(:GET, url, "application/http", "binary"; headers, spo_params(top = 3)...)
mixed = HTTP.Form([m1, m2])

batch_url = "$base_url/_api/\$batch"
auth_header = ["Authorization"  => "Bearer $token"]

req = HTTP.post(batch_url, body = mixed, headers = auth_header)
# HTTP.Messages.Response
# (a regular raw multipart response)

multiparts = HTTP.MultiPartParsing.parse_multipart(req)
# 2-element Vector{HTTP.Forms.Multipart}

responses = parse_multipart(HTTP.Response, req)
# 2-element Vector{HTTP.Messages.Response}:

results = parse_multipart(spo_parse, req)
# 2-element Vector{Vector{Any}}:

length.(results)
# 2-element Vector{Int64}:
#  2
#  3
hhaensel commented 10 months ago

I'm aware that using Forms for both mixed and form-data is not ideal. I could also think of an abstract type MultipartIO, which Form and Mixed could be subtypes of. But for the time being I didn't want to touch too much code.

hhaensel commented 2 months ago

I know, it's been a while since I submitted this one. I'd be happy to receive some feedback, in order to finilize this PR, if the functionality is considered useful.