internetarchive / warcprox

WARC writing MITM HTTP/S proxy
379 stars 54 forks source link

Add --cdxserver-dedup-cookies option #53

Closed vbanos closed 6 years ago

vbanos commented 6 years ago

It is necessary to pass cookies to the CDX Server we use for deduplication. To do this, we add the optional CLI argument --cdxserver-dedup-cookies="cookie1=val1;cookie2=val2" and if it is available, its used in the Cookie HTTP header in CDX server requests.