HAR: Inconsistent application/octet-stream post request payloads when exporting exchanges as http archive

[aluasm]

HTTPToolkit assumes all post request contents are valid UTF-8 while exporting HTTP Archive (HAR) exchanges, ignoring the presence of Content-Type: application/octet-stream headers in the process.

For example, the following binary request payload 0xa4, 0x26, 0x37, 0x13, 0x35, 0xe7, 0x50, 0x27 will fail to decode as valid UTF-8 (when reaching this constructor), and will instead transform into [0xef, 0xbf, 0xbd], 0x26, 0x37, 0x13, 0x35, [0xef, 0xbf, 0xbd], 0x50, 0x27, group [0xef, 0xbf, 0xbd] being the UTF-8 equivalent of Unicode U+FFFD, replacement character

Here's a demonstration:

Although this in it's own, can be delegated as an issue mainly affecting the HTTP Archive 1.2 spec, it is worth noting that HTTPToolkit actively mitigates this on the Response constructor equivalent, albeit, to an unknown extent.

There's been proposals on implementing an additional "encoding" field in HTTP Archive 1.3 spec, which could allow for safer handling of post request raw binary payloads, however, I was not able to find any libraries handling this out of the box. That being said, a possible temporary workaround would be to base64 encode any application/octet-stream request payload, and allow end users to work with the encoded data accordingly.

Lastly, thank you for releasing and open-sourcing such an awesome tool to the community, HTTPToolkit has been extremely helpful so far, and I look forward to finding out what interesting capabilities further development will bring.

---

Does this affect you too? Click the +:smiley: icon in the top right and add a :+1: on issues to help decide where HTTP Toolkit goes next, or go take a look & vote on the most popular ideas so far.

[aluasm]

@pimterry any updates on this?

[pimterry]

Hi @aluasm, thanks for reporting this and thanks for the PR! I'm on holiday at the moment through Christmas, but I'll take a proper look at this when I'm back in January.