haf / Http.fs

A simple, functional HTTP client library for F#
323 stars 41 forks source link

content-type header is not sent with GET request #160

Closed MarneeDear closed 5 years ago

MarneeDear commented 5 years ago

Http.fs version 5.3 and Hopac 0.3.23

My request looks like this

let request =
    Request.createUrl Get "https://www.redacted.com/Customers" //|> Request.setHeader (ContentType (ContentType.create("application", "x-www-form-urlencoded"))
    |> Request.body (BodyForm 
        [
            NameValue ("sBarcode", "11166677777")
            NameValue ("sBin", "555555")
            NameValue ("sSpan", "8888")
    ])
    |> Request.setHeader (UserAgent "Chrome or summat")
    |> Request.setHeader (ContentType (ContentType.create("application", "x-www-form-urlencoded")))
    |> Request.setHeader (Custom ("X-My-Header", "hi mum"))

I use it like this

    job {
        use! response = getResponse request
        if response.statusCode = 200 then
            let! bodyStr = Response.readBodyAsString response
            printf "%s" bodyStr
            return bodyStr
        else
            return string response.statusCode
    } |> run |> printf "WHAT HAPPENED %s"

I am pretty sure I am setting the content-type correctly, but according to the response I get from the server, I am not specifying a content-type. I also inspected the request with Fiddler and do not see the content-type in the header. I do see the other two headers, though.

Also, according to the README I should not have to specify a content-type when the body is a list of name values pairs, but this doesn't seem to be the case.

Or I am doing something wrong, maybe?

Please help, thanks.

ivpadim commented 5 years ago

@MarneeDear Please try the following instead

Request.setHeader (ContentType {typ="application"; subtype="x-www-form-urlencoded"; charset=None; boundary=None} )
MarneeDear commented 5 years ago

@ivpadim , thank you, but I am getting the same result. I inspected the request with Fiddler, and I still do not see the content-type in the header.

MarneeDear commented 5 years ago

I also do not see the body in the request.

MarneeDear commented 5 years ago

I htink http.fs does not support sending a body and content-type header in a GET request. I tried changing to a POST with everything else the same, and I see the body in the request and I see the content-type header.

I understand that it is not common practice to send a body in a GET request, but the web API I am interfacing with was designed this way for some reason.

Is there a way to get around this problem?

PostMan will let me do it.

GET /Customers? HTTP/1.1
Host: redacted.redacted.com
CustomerCode: REDACTED
Content-Type: application/x-www-form-urlencoded
User-Agent: PostmanRuntime/7.13.0
Accept: */*
Cache-Control: no-cache
Postman-Token: 899d8e97-51c1-4632-b649-f3d4d4e6e0eb,0aa73587-f958-4b1a-acd4-042e272c4883
Host: mcwapi.washassist.com
accept-encoding: gzip, deflate
content-length: 48
Connection: keep-alive
cache-control: no-cache

sBarcode=11155555+&sBin=5555+&sSpan=4255531
MarneeDear commented 5 years ago

I have resorted to using (.NET Core) System.Net.Http. The below works.

    use client = new HttpClient()
    let req = new HttpRequestMessage()
    req.Method <- HttpMethod.Get
    req.Headers.Add("CustomerCode", "REDACTED")
    req.RequestUri <- new Uri("https://REDACTED/Customers")
    req.Content <- new StringContent("sBarcode=1144445+&sBin=554443+&sSpan=44441", Encoding.UTF8, "application/x-www-form-urlencoded")
    let response = (client.SendAsync(req))
    let result = response.Result
TheAngryByrd commented 5 years ago

So it looks like this code

https://github.com/haf/Http.fs/blob/15fc0a059cf796e5f56e41f35f0f09dc8db43d15/HttpFs/HttpFs.fs#L759-L762

is preventing a body from being set on a GET request. Although unconventional, it is technically valid to send a body via a GET.

haf commented 5 years ago

@MarneeDear Sending bodies with GET requests is not supported because it "SHOULD be ignored" https://stackoverflow.com/questions/978061/http-get-with-request-body — hence it's explicitly disallowed in this lib, since this lib guides you to do the right thing.

is preventing a body from being set on a GET request. Although unconventional, it is technically valid to send a body via a GET.

No, it's not, to the app consuming the request. It's only valid to the parser consuming the request, but even then it should skip past it.

TheAngryByrd commented 5 years ago

That SO post is literally saying it's technically valid.

haf commented 5 years ago

@TheAngryByrd The the bottom of the accepted answer:

Update The RFC2616 referenced as "HTTP/1.1 spec" is now obsolete. In 2014 it was replaced by RFCs 7230-7237. Quote "the message-body SHOULD be ignored when handling the request" has been deleted. It's now just "Request message framing is independent of method semantics, even if the method doesn't define any use for a message body" The 2nd quote "The GET method means retrieve whatever information ... is identified by the Request-URI" was deleted.

seanamosw commented 5 years ago

Obviously sending a body in a GET request is wrong.

That said, I also have the joy of working with plenty bad APIs. So sometimes you just have to do these sorts of things.

A compromise could be that we add an explicit option to the request so that you can opt into this behaviour?