search

quicwg / qlog

The IETF I-D documents for the qlog format
Other
83 stars 12 forks source link

Allow raw logging of packets and/or frames #11

Closed rmarx closed 4 years ago

rmarx commented 5 years ago

As mentioned by @kazuho on the mailing list, it would be useful to (re-)introduce the ability to log raw packet and/or frame contents, probably has a hex-encoded string.

Probably easiest to add an additional field like this:

{
    "stream_type": "ACK",
    "raw_hex": "0892e340dbaa354f800239dddc7be78406fe3726bea050bb8c56ab36",
    ...
}
kazuho commented 5 years ago

Thank you for opening the issue.

Can we also omit the "stream_type" attribute, because that would be obvious from the first byte of the binary? So something like just {"raw_hex":"..."} or just the hex string itself.

rmarx commented 5 years ago

Omitting the stream_type is indeed possible. Dropping the "raw_hex" would require a move to an array (i.e., [ ]) rather than an object (i.e., { }) literal. This is something we might want to allow for issue #8 as well, so that might fit.

Just to be sure what we're talking about: We would have a mixed JSON file format with some events (e.g., recovery-related stuff) being logged in full, and the packet/frame level events being logged as binary hex strings for post-processing, right? See the example below.

"events": {
    [48, "TRANSPORT", "PACKET_RECEIVED", "DEFAULT", ["08277abefc43c25eca0892e340dbaa354f800239dddc7be78406fe3726bea050bb8c56ab36"] ],
    [49, "TRANSPORT", "FRAME_RECEIVED", "DEFAULT", ["0892e340dbaa354f800239dddc7be78406fe3726bea050bb8c56ab36"] ],
    [50, "RECOVERY", "METRIC_UPDATE", "ACK_RECEIVED", {"min_rtt": 50, "smoothed_rtt": 62} ],
}

Does that fit with what you had in mind? (I have now included both full packet and separate frame logs, obviously we could also just do the packet only)

rmarx commented 4 years ago

Lacking further follow-up on this, I've added "raw" or "raw_encrypted"/"raw_decrypted" fields where appropriate to draft-01. I do not want to require tools to deal with situations where only the "raw" fields are present (i.e., having to include a full compliant QUIC and H3 parser in each tool), so you'd have to write a separate transformer that takes the raw stuff and transforms it into "proper qlog" before putting it in a tool, but I feel it's a good compromise personally.