search

n8henrie / pycookiecheat

Borrow cookies from your browser's authenticated session for use in Python scripts.
https://n8henrie.com/2013/11/use-chromes-cookies-for-easier-downloading-with-python-requests/
MIT License
696 stars 106 forks source link

CLI defaults prevent getting chromium cookies from alternative installation methods (like snaps) #65

Open mgedmin opened 3 months ago

mgedmin commented 3 months ago

My Issue

I run pycookiecheat -u ... -b chromium and it prints {}.

Looking at the source code, I see that it hardcodes the cookie file path as being in ~/.config/chromium/Default/Cookies, which is wrong for the snap package that Ubuntu promotes. The actual cookies live in ~/snap/chromium/common/chromium/Default/Cookies (which I've verified by loading it in sqlite3 and looking at the tantalizing encrypted_value).

I imagine Firefox would have the same issue (snap package stores cookies in ~/snap/firefox/common/.mozilla/firefox/.

WHYT

(the link redirects me to https://mattgemmell.scot/books/ which doesn't seem pertinent, so I'll try to answer the "what have you tried" literally)

Please make sure you've taken these steps before submitting a new issue:

n8henrie commented 3 months ago

Thanks for bringing these to my attention

pipx run pycookiecheat --version doesn't understand the --version argument.

https://github.com/n8henrie/pycookiecheat/issues/66

(the link redirects me to https://mattgemmell.scot/books/ which doesn't seem pertinent, so I'll try to answer the "what have you tried" literally)

https://github.com/n8henrie/pycookiecheat/commit/cbfe5a641cded4df68732c77fcd0e93d5355a303

Looking at the source code, I see that it hardcodes the cookie file path

Actually, the CLI tool you're using is brand new and doesn't yet have an argument for this. For alternative installations like snaps, I think what you're looking for is the cookie_file parameter to the chrome_cookies function: https://github.com/n8henrie/pycookiecheat/blob/cbfe5a641cded4df68732c77fcd0e93d5355a303/src/pycookiecheat/chrome.py#L240C5-L240C16

You should be able to run it as something like python -c 'from pycookiecheat import chrome_cookies; print(chrome_cookies(url="https://n8henrie.com", browser="chromium", cookie_file="~/snap/chromium/common/chromium/Default/Cookies"))' (or some minor variation -- importing BrowserType to avoid the deprecation warning, possibly have to use Path("~/snap/chromium/common/chromium/Default/Cookies").expanduser(), but it seems like you should be able to take it from there.

This does seem like a useful flag to support in the CLI though!

n8henrie commented 3 months ago

https://github.com/n8henrie/pycookiecheat/issues/67

mgedmin commented 3 months ago

Actually, the CLI tool you're using is brand new and doesn't yet have an argument for this

It's a great tool! I'm not sure I would've tried to automate my VPN connection shell script if I couldn't have played a bit with pipx run pycookiecheat. If all you have is a library, well now you have to think of a name and location for a virtualenv to install that library, it's essentially starting an entire fresh new project instead of scribbling a few lines in a ~/bin/work-vpn.

I agree that a command-line argument --cookie-file would solve this. I'm also torn between the competing design principles of "thing should just work out of the box" and "in the face of ambiguity refuse the temptation to guess". Maybe the best solution (for the user) would be to have a list of common file locations and if you find multiple of those exist, print the possible --cookie-file arguments and let the user pick one. Maybe mention which one has the freshest mtime. (This is a suggestion for the command-line tool, not the library API. A stretch goal perhaps. I might be tempted to create a PR if you think it would be a good idea.)

n8henrie commented 3 months ago

If all you have is a library, well now you have to think of a name and location for a virtualenv to install that library,

True, though to be fair pipx is just doing those steps for you

$ ls -l ~/.local/pipx/venvs/ | head -5
total 0
drwxr-xr-x  7 n8henrie staff 224 Apr 23 12:22 cibuildwheel
drwxr-xr-x  7 n8henrie staff 224 Apr 23 12:23 cookiecutter
drwxr-xr-x  7 n8henrie staff 224 Apr 23 12:23 gunicorn
drwxr-xr-x  7 n8henrie staff 224 Apr 23 12:23 httpx

I agree that a command-line argument --cookie-file would solve this...

A PR would be appreciated if you have time and energy. I would expect it to "work out of the box" if only to mirror the behavior of the library, which assumes the standard system install location (at least historically has been for ubuntu / debian / arch based systems installed with the system package manager): https://github.com/n8henrie/pycookiecheat/blob/ab4937f0586f060f7435deed741f46eddcace4da/src/pycookiecheat/chrome.py#L161 . This is the current behavior.

I think adding a --list-cookie-files type option would potentially be helpful, though might also introduce a need for significantly more (and more frequent) maintenance (as additional installation options / locations are developed, as reqeusts to support additional Chrome-based browsers continue).