Closed aqw closed 3 years ago
Ah well I see that these are actually redundant.
But as the auth header gets an authentication token added,
I am curious if it makes a difference when I pass the header with simply
Authorization: bla bla
.
Does this then still have all the other defaults like Connection:keep-alive
?
If it does no problem.
I'm unsure. You'll need to read requests
s docs to see what they're doing by default. But agree the majority, if not all, of those headers are unnecessary.
It seems that is works without all the headers. Good to know that default headers exist
I did not find any issues with that so I guess we can close this issue again.
In
scraper_globals.py
, the headers for bothLOGIN_
andAUTH_
are hardcoded. It is considered poor form to spoof the headers unnecessarily.Unless they are sniffing for user agents (which would be worthy of a snarky comment in the source code), nothing in those headers looks atypical. Ideally, they can all be removed and the defaults used.