How does WARCreate handle preserving HTTP/2 communication?

machawk1 / warcreate

Chrome extension to "Create WARC files from any webpage"

https://warcreate.com

MIT License

205 stars 13 forks source link

How does WARCreate handle preserving HTTP/2 communication? #103

Open machawk1 opened 6 years ago

machawk1 commented 6 years ago

The current behavior needs to be documented. Per the WARC/1.1 spec, there is no documented "right way" but identifying the current approaches would be useful and help to guide how it might be done.

ibnesayeed commented 6 years ago

For the reference https://github.com/iipc/warc-specifications/issues/15

machawk1 commented 6 years ago

That discussion was part of the impetus for creating this issue, @ibnesayeed. This issue is about determining what the current behavior is. The result of that can be cross-referenced with other discussions on the correct representation.

N0taN3rd commented 6 years ago

@machawk1 I would assume the "right" way would be to turn the HTTP/2 protocols into HTTP/1.1.

Reasoning:

currently no existing replay system handles replay of HTTP/2
tools like warcreate and Squidwarc see headers as HTTP/2 but see the bodies as if they were sent over HTTP/1.1 (not the individual parts of the stream)

machawk1 commented 6 years ago

@N0taN3rd Maybe a conversion record is more suitable for the HTTP/1.1 derivative while still maintaining the original payload while replay systems (and the WARC spec) catch up.

ibnesayeed commented 6 years ago

I would agree with @machawk1, because as an archivist you never want to lose information that might be useful later just because current tools do not support something.