Closed sebastian-nagel closed 7 years ago
If a HTTP header field is empty in the WARC response record, the WAT extractor does not use the empty value but uses the field value from the preceding HTTP header field.
WARC record HTTP header with empty "Server" name:
HTTP/1.1 200 OK Content-Type: text/html; charset=utf-8 Content-Length: 28676 ... Expires: Fri, 28 Apr 2017 14:46:35 GMT Server: Set-Cookie: FixedFacetDefaults=None; path=/ ...
WAT record extracted, "Server" name set to "Expires" date:
"Envelope": { "Format": "WARC/1.0", ..., "Payload-Metadata": { "Actual-Content-Type": "application/http; msgtype=response", "HTTP-Response-Metadata": { "Headers-Length": "1092", ..., "Headers": { "X-Cache": "Miss from cloudfront", "Server": "Fri, 28 Apr 2017 14:46:35 GMT", "Connection": "keep-alive", ..., "Expires": "Fri, 28 Apr 2017 14:46:35 GMT", ...,
Reproduced and verified using empty_header_field.warc.gz.
If a HTTP header field is empty in the WARC response record, the WAT extractor does not use the empty value but uses the field value from the preceding HTTP header field.
WARC record HTTP header with empty "Server" name:
WAT record extracted, "Server" name set to "Expires" date: