Robust handling of HTTP header values

[c0c0n3]

We should be able to decode/encode correctly any HTTP header value as specified by section 3.2.6. We need this for processing the JSON containing the IDS token which gets encoded as an HTTP quoted string in the IDS header---see this example.

Ideally we'd use an HTTP lib to do this, but I couldn't manage to get it right with textproto, http & friends---i.e. Go's standard approach to handling HTTP messages. So at the moment we have a stopgap solution in place to decode quoted values which is okay-ish for a prototype but not really what you'd like to have for a production grade product.

Here's a possible way forward:

• Scrap current poor man's implementation of parseIdsHeaderValue in sec/tokenrep.go
• See if you can implement it using Go's built in libs---see notes below.
• Failing that, comb the interwebs to find a lib that implements robust handling of HTTP headers as spec'd by RFC 7230.

Another way to look at it is that we could perhaps change the way the JSON header value gets encoded. For example, wouldn't it be easier to encode the JSON payload in Base 64?

"IDS header example"

"HTTP - Field Value Components"

[c0c0n3]

Go built-in HTTP headers don't support quoted values?!

Is that so or am I doing something horribly wrong? According to the interwebs, the below code should be able to handle a header value that's a quoted string, e.g. "{\"url\": \"http:\/\/golang\"}"

func parseIdsHeaderValue(rawValue string) (parsedValue string, err error) {
    headerName := "header"
    rawHeader := fmt.Sprintf("%s: %s\n", headerName, rawValue)
    buf := bufio.NewReader(strings.NewReader(rawHeader))
    parser := textproto.NewReader(buf)

    h, err := parser.ReadMIMEHeader()
    if err != nil {
        return "", err
    }
    parsedValue = textproto.MIMEHeader(h).Get(headerName)
    return parsedValue, nil
}

See e.g. https://stackoverflow.com/questions/22561661/ But when I test it, I get some weirdness I can't explain:

func TestMystery(t *testing.T) {
    want := `{"url": "http://golang"}`
    raw := `"{\"url\": \"http:\/\/golang\"}"`
    if got, err := parseIdsHeaderValue(raw); err != nil {
        t.Errorf("%s", err)
    } else if got != want {
        t.Errorf("wrong! got: %s, want: %s", got, want)
    }
}

the result of calling ReadMIMEHeader on this line

h, err := parser.ReadMIMEHeader()

is

h   = [0] -> `"{\"url\": \"http:\/\/golang\"}"`
err = EOF

i.e. the parse fails but the parse result gets built anyway, pity it's not what RFC 7230 says it should be :-O

"HTTP - Field Value Components"

[gboege]

@c0c0n3 Please see my reply here in favor of dropping the escaping and prefering to Base64 encode.

[c0c0n3]

we decided to encode the IDS header value as Base 64 rather than HTTP quoted value---see comment to #7.

Closing this issue as it's no longer relevant.