rs / curlie

The power of curl, the ease of use of httpie.
https://curlie.io
MIT License
2.86k stars 97 forks source link

Same command works in curlie but not in curl #63

Closed lmdamato closed 1 year ago

lmdamato commented 1 year ago

I am trying to run a POST request in vanilla cURL, but failing at it. I can easily run it in curlie (I expect the 500):

$ curlie -v -H 'X-SLURM-USER-TOKEN: someLongToken' -H 'X-SLURM-USER-NAME: lmdamato' -X POST -d '{"job":{"environment":{"LD_LIBRARY_PATH":"/lib/:/lib64/:/usr/local/lib","PATH":"/bin:/usr/bin/:/usr/local/bin/"},"name":"my_test_hello_world_job","partition":"QT","qos":"medium"},"script":"#!/bin/bash\n/bin/hostname"}' http://myslurmhost:6820/slurm/v0.0.37/job/submit
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to myslurmhost (127.0.0.1) port 6820 (#0)
POST /slurm/v0.0.37/job/submit HTTP/1.1
Host: myslurmhost:6820
User-Agent: curl/7.61.1
X-SLURM-USER-TOKEN: someLongToken
X-SLURM-USER-NAME: lmdamato
Content-Type: application/json
Accept: application/json, */*
Content-Length: 217

{
    "job": {
        "environment": {
            "LD_LIBRARY_PATH": "/lib/:/lib64/:/usr/local/lib",
            "PATH": "/bin:/usr/bin/:/usr/local/bin/"
        },
        "name": "my_test_hello_world_job",
        "partition": "QT",
        "qos": "medium"
    },
    "script": "#!/bin/bash\n/bin/hostname"
}

* upload completely sent off: 217 out of 217 bytes
HTTP/1.1 500 INTERNAL ERROR
Connection: Close
Content-Length: 385
Content-Type: application/json

* Closing connection 0
{
    "meta": {
        "plugin": {
            "type": "openapi\/v0.0.37",
            "name": "Slurm OpenAPI v0.0.37"
        },
        "Slurm": {
            "version": {
                "major": 21,
                "micro": 7,
                "minor": 8
            },
            "release": "21.08.7"
        }
    },
    "errors": [
        {
            "error_code": 5005,
            "error": "Zero Bytes were transmitted or received"
        }
    ]
}

But the exact same command in vanilla cURL won't work:

$ curl -v -H 'X-SLURM-USER-TOKEN: someLongToken' -H 'X-SLURM-USER-NAME: lmdamato' -X POST -d '{"job":{"environment":{"LD_LIBRARY_PATH":"/lib/:/lib64/:/usr/local/lib","PATH":"/bin:/usr/bin/:/usr/local/bin/"},"name":"my_test_hello_world_job","partition":"QT","qos":"medium"},"script":"#!/bin/bash\n/bin/hostname"}' http://myslurmhost:6820/slurm/v0.0.37/job/submit
Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to myslurmhost (127.0.0.1) port 6820 (#0)
> POST /slurm/v0.0.37/job/submit HTTP/1.1
> Host: myslurmhost:6820
> User-Agent: curl/7.61.1
> Accept: */*
> X-SLURM-USER-TOKEN: someLongToken
> X-SLURM-USER-NAME: lmdamato
> Content-Length: 217
> Content-Type: application/x-www-form-urlencoded
> 
* upload completely sent off: 217 out of 217 bytes
< HTTP/1.1 400 BAD REQUEST
< Connection: Close
< Content-Length: 22
< Content-Type: text/plain
< 
* Closing connection 0
Unable to parse query.

Can someone help me understand why?

lmdamato commented 1 year ago

Found a solution to my own issue, thanks to a combination of two factors:

  1. there are inconsistencies between how curlie and curl handle POST requests. This only happens when passing data using the -d option (but not when passing --data!). In fact, my same curlie command, using --data instead of -d, fails. This is a documented issue: https://github.com/rs/curlie/issues/11
  2. Fortunately, curlie has a nice --curl option, which prints out the curl command instead of executing it.

I was able to infer that, in specific cases (like the one mentioned in 1. above), curlie injects additional Content-Type and Accept headers in the request, and thanks the extra headers the request succeeded.

In my specific case, it was just a matter of adding -H 'Content-Type: application/json' -H 'Accept: application/json, */*' to the curl command:

curl -v -H 'Content-Type: application/json' -H 'Accept: application/json, */*' -H 'X-SLURM-USER-TOKEN: someLongToken' -H 'X-SLURM-USER-NAME: lmdamato' -X POST -d '{"job":{"environment":{"LD_LIBRARY_PATH":"/lib/:/lib64/:/usr/local/lib","PATH":"/bin:/usr/bin/:/usr/local/bin/"},"name":"my_test_hello_world_job","partition":"QT","qos":"medium"},"script":"#!/bin/bash\n/bin/hostname"}' http://myslurmhost:6820/slurm/v0.0.37/job/submit