add feature to save RSS file, and headers, to disk

[rahulbot]

If the SAVE_RSS_FILES env-var is set to 1, then for each queued job two files will be created:

• [FEED_ID]-summary.json - summary info and headers
• [FEED_ID]-content.txt - the actual full RSS text

Sample summary JSON:

{
    "id": 133746,
    "url": "https://www.destentor.nl/buitenland/rss.xml",
    "mcFeedsId": 874977,
    "mcMediaId": 38933,
    "statusCode": 200,
    "headers": {
        "Content-Type": "application/rss+xml;charset=UTF-8",
        "ETag": "W/\"09e5bea5a36ddee446052eafc7484536f\"",
        "X-Content-Type-Options": "nosniff",
        "X-XSS-Protection": "1; mode=block",
        "Strict-Transport-Security": "max-age=31536000 ; includeSubDomains",
        "Referrer-Policy": "same-origin",
        "X-Frame-Options": "DENY",
        "Content-Encoding": "gzip",
        "Content-Length": "8384",
        "Expires": "Mon, 11 Jul 2022 19:42:23 GMT",
        "Cache-Control": "max-age=0, no-cache, no-store",
        "Pragma": "no-cache",
        "Date": "Mon, 11 Jul 2022 19:42:23 GMT",
        "Connection": "keep-alive",
        "Vary": "Accept-Encoding",
        "Link": "<https://images0.persgroep.net>; rel=preconnect;"
    }
}

[rahulbot]

Would it be more useful to name it by mc-feeds-id?

[philbudne]

Yeah, I think mc_feeds_id would be better, at least right now!

[rahulbot]

OK. Any other changes or should I merge/push this and let it run for a while to collect some data?

[philbudne]

No additional requests, plenty of metadata to peruse... Thanks!

[philbudne]

(so yes, merge and run)!