Nested JSON with property name containing colon interpreted as header declaration

[ingemaradahl]

The Nested JSON CLI syntax allows me to construct a request body without having to type out the raw JSON (loving it), but it seems to collide with how arguments can be interpreted as headers. I'm expecting http POST httpie.io/hello foo[bar:baz]=foobar --print=HB and http POST httpie.io/hello foo:='{"bar:baz": "foobar"}' --print=HB to produce identical results, but when using the nested JSON syntax, httpie interprets my argument as the foo[bar header (which is an invalid HTTP header field name) having the baz]=foobar value.

$ http POST httpie.io/hello foo[bar:baz]=foobar --print=HB
POST /hello HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 0
Host: httpie.io
User-Agent: HTTPie/3.2.1
foo[bar: baz]=foobar

should produce the same result as

$ http POST httpie.io/hello foo:='{"bar:baz": "foobar"}' --print=HB
POST /hello HTTP/1.1
Accept: application/json, */*;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 30
Content-Type: application/json
Host: httpie.io
User-Agent: HTTPie/3.2.1

{
    "foo": {
        "bar:baz": "foobar"
    }
}

I think it's reasonable that nested JSON object specification on the CLI should take precedence over the intent to specify an http header, since the latter would be illegal and not accepted by many HTTP servers. For reference, xh also contains this problem, but doesn't attempt to make the request:

$ xh POST httpie.io/hello foo[bar:baz]=foobar --print=HB
xh: error: invalid HTTP header name

[jkbrzt]

Please see: https://httpie.io/docs/cli/escaping-rules

[ingemaradahl]

Thanks for looking, @jakubroztocil

Unfortunately, the escaping rules won't help:

$ http --print=HB POST httpie.io/hello -- foo[bar:baz]=foobar
POST /hello HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 0
Host: httpie.io
User-Agent: HTTPie/3.2.1
foo[bar: baz]=foobar

httpie still interprets foo[bar:baz]=foobar as an illegal header declaration.

[ingemaradahl]

Double escaping : results in the expected behavior, but due to the default behavior creating invalid header names, it shouldn't be required, really:

$ http POST httpie.io/hello foo[bar\\:baz]=foobar --print=HB
POST /hello HTTP/1.1
Accept: application/json, */*;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 30
Content-Type: application/json
Host: httpie.io
User-Agent: HTTPie/3.2.1

{
    "foo": {
        "bar:baz": "foobar"
    }
}

[jkbrzt]

Double escaping : results in the expected behavior,

HTTPie only needs one backslash — the second one is for the shell. You can single-quote the argument to avoid the second backslash (at least for Bash):

$ http POST httpie.io/hello 'foo[bar\:baz]=foobar' --print=HB

but due to the default behavior creating invalid header names, it shouldn't be required, really

The key/value request items parsing uses the separator to determine the item type. Checking for effective validity on top of that would be an interesting heuristic for improved UX.

However, this semantic context sensitivity would make the language much less predictable (and harder to parse). It’s also not always clear-cut what should be considered valid.

We also generally allow you to specify weird/invalid requests for testing purposes; it’s an intentional feature, but sometimes it hits the limits of the underlying network libraries (like with this header).

[ingemaradahl]

Sure, but httpie is primarily a CLI utility for most people (I assume), and its popularity is most likely based on its ease of use (in comparison to cURL). You're suggesting I add three additional characters (one which also require a modifier key to type in) to make the request, which IMO goes against ease of use.

I haven't looked at the parsing solution in httpie, but this is pretty much akin to operator precedence. = should have a higher precedence than :. This shouldn't really make the parsing code more complex, but again, I haven't looked at it. My point is that I don't buy the "parsing would become complex" argument.

I also don't buy the "it's not a bug, it's a feature" reasoning. It's more difficult to "do the right thing" than to "do the really non-standard edge case which might be usable for testing thing" which is a shame.