n8henrie / pycookiecheat

Borrow cookies from your browser's authenticated session for use in Python scripts.
https://n8henrie.com/2013/11/use-chromes-cookies-for-easier-downloading-with-python-requests/
MIT License
744 stars 111 forks source link

Doesn't return session cookies #71

Closed brianjmurrell closed 1 month ago

brianjmurrell commented 1 month ago

My Issue

One particular cookie being set is not being returned by chrome_cookies()

I have the following code:

            cookies = chrome_cookies(f"https://secure.[redacted].com", browser=BrowserType.CHROMIUM)
            for cookie in cookies.keys():
                print(f"Found cookie {cookie} = {cookies[cookie]} for secure.example.com")

but it is failing to return the session-id cookie which you can see is being set by the website in this devtools response:

image

Additionally, in devtools->Application tab we can see the cookie is present for the site:

image

pycookiecheat doesn't return that cookie either but it returns a dozen or so others.

WHYT

I don't know what to try other than the above code snippet and pycookiecheat CLI.


Please make sure you've taken these steps before submitting a new issue:

n8henrie commented 1 month ago

That's a session cookie. Because session cookies don't persist beyond a session, I presume they never leave your browser's memory and are never written to disk, which means they won't be retrievable by this tool (or likely any other tool that can't read your browser's memory). If you need access to these, I suggest you look into browser extensions instead of command-line tools.

brianjmurrell commented 1 month ago

Yet another attempt at trying to get data from this particular website failed. :frowning_face:

I suggest you look into browser extensions instead of command-line tools

Fair enough. Any suggestions? I'd want to write the cookies (or even just the page content) out on every page refresh (I also employ a timed-page reloader).

Quite honestly, I'd prefer (and did attempt) to just log into the website using Python requests (like I do for a number of others) but this particular website has that Akamai Bot Detector garbage in front of it so I need the browser to establish the session and use an extension to reload the page I want to keep the session alive. And as I said, I'd be just as happy to have the page written to a file on page reload even.

n8henrie commented 1 month ago

Any suggestions?

Nothing terribly specific, but if you're familiar with JavaScript, you might be able to work something out with GreaseMonkey / TamperMonkey (I'm no good at JS though, so this could be a dead end).

For similar tasks I've used playwright in the past. If you haven't tried it, it's worth a look.