I can't get_cookies() even with browser.privatebrowsing.autostart=false

[t0m4s]

Hi ! First of all, thank you for the work you've done with tbselenium !

I'm trying to save my cookies after logging to a website (https, no onion). I saw the issue #79 and tried to change the private browsing mode but I still can't get any cookie.

Here is the pref_dict passed to my tbselenium instance :

pref_dict = { "browser.privatebrowsing.autostart": False }
driver = TorBrowserDriver(TBB_DIR, socks_port=socks_port, control_port=control_port, tor_cfg=cm.USE_STEM, executable_path=GECKO, tbb_logfile_path='/dev/null', options=options, pref_dict=pref_dict)

I also tried this: print(driver.execute_script("return document.cookie;")) but it returns only the first cookie variable.

If I use add_cookie() to add dummy data, I'm able to retrieve it with get_cookies() :

driver.add_cookie({'name' : 'foo', 'value' : 'bar'})
print(driver.get_cookies())

My dummy data is retrieved but no other cookie data : [{'name': 'foo', 'value': 'bar', 'path': '/', 'domain': 'www.google.fr', 'secure': False, 'httpOnly': False, 'sameSite': 'None'}]

I'm using the latest tbselenium version. Could somebody else test if "browser.privatebrowsing.autostart": False is still working ? Any idea why the javascript trick is only returning part of the cookie ?

Thanks in advance for your answers :-).

[t0m4s]

Update :

I'm now able to get all of the cookies via javascript. I needed to add the website to the whitelist in TBB : Edit -> Settings -> Privacy & security -> Cookies and site data -> Manage exceptions...

But I'm still not able to restore my session from my cookies. I know it should be ok, I've already been able to do it on the same website, using selenium through a TOR instance. Here is what I'm doing to try and restore the cookies :

driver.load_url(base_url)
driver.delete_all_cookies()
for each cookie :
            driver.add_cookie({'name': name, 'value': value})
print(driver.get_cookies())
driver.refresh()
print(driver.get_cookies())

The cookies seem to be present. I've checked them via the Web developer tools -> storage -> cookies and the to print() before and after refreshing in the code above. But no way to restore the session.

I know we're not on a help forum but if anybody as a suggestion, I'll take it.

[t0m4s]

Solution found : saving and restoring cookies only by using javascript : cookie = driver.execute_script("return document.cookie;") driver.execute_script("document.cookie = '" + cookie + "';")

Don't know why but driver.get_cookies() and driver.add_cookie() seem to manage cookies which aren't taken in account by the browser.

[gunesacar]

Hi @t0m4s , I'm glad you found a solution. I'm really not sure what's going on there.

However, there are two reasons why document.cookie may not return all the cookies you see in the devtools/storage/browser interface: 1) it wouldn't return HttpOnly cookies 2) it wouldn't return cookies from different browsing contexts (e.g. iframes with a different origin)

I probably won't have time to do a through investigation anytime soon. So I'm glad you've found a way out.

[t0m4s]

Update : solution not found editing Edit -> Settings -> Privacy & security -> Cookies and site data -> Manage exceptions... can't have any effect on my problem as tbselenium is using its own profile. I can't get cookies anymore and I can't figure out what I've could have change. I'll let you know if I finally find what's going wrong.

[t0m4s]

Fatal error detected module Brain is not responding restarting Human in fail-safe mode...

I was using broken code to restore my saved cookies... Abstract from all of my tests : "browser.privatebrowsing.autostart": False has no effect on driver.get_cookies() editing Edit -> Settings -> Privacy & security -> Cookies and site data -> Manage exceptions... can't have any effect as tbselenium is using its own profile. save your cookies using cookies = driver.execute_script("return document.cookie;") restore your cookies with :

for cookie in cookies.split("; "):
    driver.execute_script("document.cookie = '" + cookie + "';")

Sorry for inconvenience.

[rewiaca]

So there are no method to get HttpOnly cookies from onion website?

[gunesacar]

So there are no method to get HttpOnly cookies from onion website?

If you can get HttpOnly cookies in selenium (using FirefoxDriver), but not in tbselenium, please open an issue and I'll try to look into it.