Raspberry Pi 3+ -> Error occured while deleting cookies

silensinator commented 5 years ago

Expected Behavior

To run the bot normally

Current Behavior

Just 4 hours ago hit with the road bump of the following:

Error occur while deleting cookies

Tried: -> Reinstall instapy -> Reinstall chromedriver -> Reboot

Possible Solution (optional)

InstaPy configuration

-> Running with raspberry pi 3+

malimccalla commented 5 years ago

I think its an issue with chromedriver 2.36

python3 -m pip install --user instapy-chromedriver

That will upgrade you to latest version

silensinator commented 5 years ago

@malimccalla Not sure if I am confused. But as far as I know, raspberry Pi does not use the same instapy-chromedriver right?

Refer to: https://github.com/InstaPy/instapy-docs/blob/master/How_Tos/How_to_Raspberry.md

python3 -m pip uninstall instapy-chromedriver

I already executed: python3 -m pip install --user instapy-chromedriver==2.36.post0 cp /home/pi/.local/lib/python3.5/site-packages/instapy_chromedriver/chromedriver assets/

But the error still occurring. What could be the problem?

silensinator commented 5 years ago

I tried the basic quickstart.py also facing this issue:

silensinator commented 5 years ago

will it caused by this:

malimccalla commented 5 years ago

I had the same issue, it just suddenly stopped working. Once I upgrade instapy-chromedriver it worked fine. Try running the command I put above on your pi and see if it fixes your problem

silensinator commented 5 years ago

Hi @malimccalla are you using raspberry pi?

I followed your instruction. Nope still same error. I changed my password as well but the session feedback was:

malimccalla commented 5 years ago

Yes I'm using raspberry pi. In your above screenshot it says your chromedriver version is still 2.36 - please run this instead

pip3 install instapy-chromedriver -U

silensinator commented 5 years ago

@malimccalla tried but still same:

Wonder what causes the page crash @uluQulu @timgrossmann any idea?

malimccalla commented 5 years ago

Your version is still 2.36 - I upgraded from that version to 2.46 and everything started working again. If you figure out how to upgrade your instapy-chromedriver version it will work.

silensinator commented 5 years ago

@malimccalla how do you get 2.46? Could you navigate to /home/pi/.local and get inside the python 3.5 and send the chromedriver here?

@uluqulu where could I get the latest chromedriver for the /asset folder in raspberry pi?

silensinator commented 5 years ago

Okay guys, it is not about the chrome version or anything like it. It is because of the file permission. All you need to do is:

sudo chmod 777 \path\to\your\chromedriver

where the \path could be pointing towards your assets (those who are using standalone).

Side discovery: To update your chrome driver to the latest version sudo apt-get install chromium-driver

copy the "chromedriver" from /usr/bin to your asset folder

silensinator commented 5 years ago

Okay guys, now the thing occurs intermittently where 90% crash and 10% okay >< @malimccalla yours was stable?

xerZV commented 5 years ago

Same error here - raspberry pi 3+

silensinator commented 5 years ago

After formatting raspberry pi and mix&match chromedriver for hours and result is still not expected,

Discovered a temporary bypass:

add to your session use_firefox=True

install firefox sudo apt-get install firefox-esr In your raspberry pi download this: https://github.com/mozilla/geckodriver/releases/download/v0.17.0/geckodriver-v0.17.0-arm7hf.tar.gz or shorter link: https://bit.ly/2HdfWu5

extract geckodriver from the above file to /usr/bin

Run your script.

PS: headless option is not working for firefox even headless_browser=True any idea @uluQulu @timgrossmann @converge ?

PS2: It will use more resources than usual as the headless browser option is not functioning

converge commented 5 years ago

@silensinator you can use nogui=True , maybe you will also need to set the window size in browser.py:

if use_firefox:
 44         firefox_options = Firefox_Options()
 45
 46         if headless_browser:
 47             firefox_options.add_argument('-headless')
 48
 49         if browser_profile_path is not None:
 50             firefox_profile = webdriver.FirefoxProfile(
 51                 browser_profile_path)
 52         else:
 53             firefox_profile = webdriver.FirefoxProfile()
 54
 55         # set window size
 56         firefox_options.add_argument('--width=1920')
 57         firefox_options.add_argument('--height=1080')

silensinator commented 5 years ago

@converge appreciate your shift reply.

Not sure is it raspberry pi 3 issue. I tried:

adjusting the width/height in browser.py
incorporating nogui=True

But those actions (either/and) will crash/abrupt close of firefox when it tries to crawl the follower's list

For now I will just leave it as default:

(username=insta_username,
password=insta_password,
 use_firefox=True,
#nogui=True,
headless_browser=True,
disable_image_load=True)

dontic commented 5 years ago

@silensinator well after days of research I finally found what I think is the solution. All thanks to the great @JoeFertig

For now I have 2 instances running and they haven't crashed yet. I'll keep updating.

New updated Raspberry Pi 3b & 3b+ guide HERE

If you had a previous version of InstaPy with 2.36 chromedriver you just need to do the following:

Uninstall chromedriver 2.36:

python3 -m pip uninstall instapy-chromedriver
Delete chromedriver from /InstaPy/assets folder

Install new chromedriver 3.0.0 beta version:

cd InstaPy
cd assets
wget https://github.com/electron/electron/releases/download/v3.0.0-beta.5/chromedriver-v3.0.0-beta.5-linux-armv7l.zip
unzip chromedriver-v3.0.0-beta.5-linux-armv7l.zip
chmod 755 chromedriver
chmod +x chromedriver
rm chromedriver-v3.0.0-beta.5-linux-armv7l.zip

Joker6888 commented 5 years ago

Tried it out because I have the same problem, no solution here. Once my unfollow-action ran good, then the next day same problem again.

jeremycjang commented 5 years ago

Followed @dontic 's steps , testing

dontic commented 5 years ago

Followed @dontic 's steps , testing

Update on my side. One of my two instances crashed with :

Error occurred while deleting cookies from web browser! b'Message: unknown error: session deleted because of page crash\nfrom tab crashed\n (Session info: headless chrome=72.0.3626.121)\n (Driver info: chromedriver=2.36 (a81166ad79e68fbfe7cf5ba243192d6412e26b37),platform=Linux 4.14.98-v7+ armv7l)

After running it again it has been running fine until now.

Don´t know why it still says:

chromedriver=2.36

Will keep testing.

jeremycjang commented 5 years ago

@dontic Also crashed for me today after a few successful instances run consecutively.

Not sure if this might be related/what the design decision was surrounding going with version 2.36:

https://bugs.chromium.org/p/chromedriver/issues/detail?id=2333

JoeFertig commented 5 years ago

@silensinator well after days of research I finally found what I think is the solution. All thanks to the great @JoeFertig

Hello & thanks! Well the main problem is that you need a certain chromedriver for a certain chromium-browser. Most people update the chromedriver to a version that is not matching with the chromium-browser.

In our case ("unkown SessionID" & "error while deleting cookies" etc) I did some tests cause I still experience this problem. So the best workaround atm is to remove chromium-browser and re-install it. You can also try and delete all the browser data manually from the chromium-browser.

I think the main problem is Rasps amrV7 chip and the lack of developement for that chipset. My solution is to move from a Pi to another micro computer which has no arm chips implemented.

Joker6888 commented 5 years ago

Thank you for your information. Is there a page where I can see which version of Chromium and Chromedriver matches together so I can install both fitting versions of it? The problem on the raspberry pi is killing me right now. Which micro computer do you recommend for the Instapy project?

dontic commented 5 years ago

@JoeFertig thanks again! If it comes to that I will definitely check out some other microcomputers.

For now I'm testing another possible solution, can't say if it works yet, will update in a day or so.

Im going to put it here if anyone wants to test it (@jeremycjang @silensinator @Joker6888):

Uninstall chromium 72: sudo apt-get purge chromium-browser rm -rf ~/.config/chromium
Install chromium 65 (because of https://github.com/timgrossmann/InstaPy/issues/4033#issuecomment-466074266): wget http://launchpadlibrarian.net/361669485/chromium-browser_65.0.3325.181-0ubuntu0.14.04.1_armhf.deb; sudo dpkg -i chromium-browser_65.0.3325.181-0ubuntu0.14.04.1_armhf.deb
Install codecs: wget http://launchpadlibrarian.net/361689926/chromium-codecs-ffmpeg_65.0.3325.181-0ubuntu0.16.04.1_armhf.deb; sudo dpkg -i chromium-codecs-ffmpeg_65.0.3325.181-0ubuntu0.16.04.1_armhf.deb
Use this with chromedriver 2.36 (install it like in the RPi tutorial)

Hope this is the right match between chromium and chromedriver. It's been working fine now but I'd like to give it some time.

jeremycjang commented 5 years ago

@dontic I'm currently testing with chromium 65 and instapy-chromedriver 2.36.post0 .

If a chromedriver is present in the assets folder, it'll use that chromedriver instead of the instapy-chromedriver that was built for the pi, so I deleted the previous chromedriver to get the program to use instapy-chromedriver .

It's working well so far, but will continue testing as well. Looks like for whatever reason, I already had the 3.0.0 beta version stored in the assets folder of the workspaces the program was pulling from!

JoeFertig commented 5 years ago

Here is a compatibility list of chromedrivers and chrome(*ium) browsers:

http://chromedriver.chromium.org/downloads

However, my solution is more a workaround than a real fix.

Joker6888 commented 5 years ago

@dontic seems to work, had to install matching codec but now it’s running again with chromium 65 and 2.36/0. Will keep testing it.

dontic commented 5 years ago

@Joker6888 I also installed the codecs, forgot to say that, edited in my previous post.

Joker6888 commented 5 years ago

Crashed after 16 minutes with unknown session ID. Any ideas?

JoeFertig commented 5 years ago

Crashed after 16 minutes with unknown session ID. Any ideas?

did you remove and re install chromium-browser?

dontic commented 5 years ago

@Joker6888 yup mine just crashed too. The only difference I can see is that now it doesn't crash because of the cookies problem:

Error message

> OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO > > INFO [2019-05-21 17:14:58] [] Session ended! > oooooooooooooooooooooooooooooooooooooooooooooooooooooo > > > > Traceback (most recent call last): > File ".py", line 113, in > session.like_by_tags(random.sample(like_tag_list, 3),amount=random.randint(15, 50), interact=True) > File "/home/pi/.local/lib/python3.5/site-packages/instapy/instapy.py", line 1877, in like_by_tags self.user_interact_media) > File "/home/pi/.local/lib/python3.5/site-packages/instapy/instapy.py", line 1967, in like_by_users media) > File "/home/pi/.local/lib/python3.5/site-packages/instapy/like_util.py", line 408, in get_links_for_username > if not is_page_available(browser, logger): > File "/home/pi/.local/lib/python3.5/site-packages/instapy/util.py", line 1658, in is_page_available page_title = get_page_title(browser, logger) > File "/home/pi/.local/lib/python3.5/site-packages/instapy/util.py", line 1720, in get_page_title explicit_wait(browser, "PFL", [], logger, 10) > File "/home/pi/.local/lib/python3.5/site-packages/instapy/util.py", line 1547, in explicit_wait result = wait.until(condition) > File "/home/pi/.local/lib/python3.5/site-packages/selenium/webdriver/support/wait.py", line 71, in until > value = method(self._driver) > File "/home/pi/.local/lib/python3.5/site-packages/instapy/util.py", line 1535, in "return document.readyState") > File "/home/pi/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 636, in execute_script > 'args': converted_args})['value'] > File "/home/pi/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute > self.error_handler.check_response(response) > File "/home/pi/.local/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response > raise exception_class(message, screen, stacktrace) > selenium.common.exceptions.WebDriverException: Message: unknown error: unknown sessionId > (Session info: headless chrome=65.0.3325.181) > (Driver info: chromedriver=2.36 (a81166ad79e68fbfe7cf5ba243192d6412e26b37),platform=Linux 4.14.98-v7+ armv7l)

Right now I'm looking at alternatives to replace the Raspberry

Joker6888 commented 5 years ago

@joeFertig yes i did, removed everything and reinstalled. Error log shows the right version (65 & 2.36)

JoeFertig commented 5 years ago

That's pretty bad. I really don't get why we get this error more and more. I tried firefox but somehow it's not able to log in. Looks like it stops typing the complete password when trying.

Joker6888 commented 5 years ago

Made a few testings, it seems to work with the 65 Chromium and beta 3 chromedriver. 2 runs just work fine, keep on testing.

jeremycjang commented 5 years ago

My Pi crashed and I had to do a completely fresh install following the latest How to Raspberry Pi guide:

https://github.com/InstaPy/instapy-docs/blob/master/How_Tos/How_to_Raspberry.md

The session now crashes after navigating to instagram.com with an error implying an empty response, though a response was returned successfully.

Switching to testing in FireFox for now to confirm if it's a good alternative while the crashing due to chromedriver gets sorted out

jeremycjang commented 5 years ago

@Joker6888 @dontic @JoeFertig the latest version of NOOBS came with Chromium 72, but it doesn't work running with chromedriver 2.36.post0 nor the supposedly compatable chromedriver 2.46 .

Can you share the linux command to install Chromium 65? sudo apt-cache madison chromium-browser only returns Chromium 72

thanks!

JoeFertig commented 5 years ago

Hey @jeremycjang does firefox work for you? Is it stable? I can't get it run.

JoeFertig commented 5 years ago

For everyone who has the 'unknown SessionID' or the 'error deleting cookies from browser' problem please try this:

sudo apt install chromium-browser chromium-codecs-ffmpeg

then run again and tell me if it fixed your problems.

dontic commented 5 years ago

Ok guys so after that initial crash it hasn't crashed since. It's been 2 days and it has been fine, no errors on my 2 instances.

@jeremycjang You can refer to my previous post https://github.com/timgrossmann/InstaPy/issues/4441#issuecomment-494371736

I think, as @JoeFertig says, that is the codecs that do the trick.

Will keep updating with any errors.

To summarise: For now Chromium 65 and chromedriver 2.36 post0 is the combination that's working for me.

jeremycjang commented 5 years ago

I think the Raspberry Pi How to documentation should be updated with that. Not familiar with how to do that on GitHub or how to modify the code now that the whole program's through a single pip install , but happy to do so if I learn how!

jeremycjang commented 5 years ago

@JoeFertig FireFox-esr was working; I was never able to get normal FireFox to work on the RPi.

However, commenting from FireFox-esr leaves a � at the end of every comment due to how the ARMS chip interacts with the encodings chosen for InstaPy. It's usable, but reading through https://github.com/timgrossmann/InstaPy/issues/1411 and https://github.com/timgrossmann/InstaPy/issues/1740 it sounds like you would have to tweak the core code to fix it, which would mess with how the commenting works when using Chromedriver

Essentially, you'd have to make a choice between which browser you primarily want to use for running the program and make the tweak for commenting depending on your choice. Personally, I only encountered the crash in this issue when using the like-based actions, so I'm running firefox-esr for like-only programs

Still testing to see if it can complete a firefox-esr like_by_location + like_by_tags run without freezing or crashing, will provide an update!

JoeFertig commented 5 years ago

@dontic nice! can you send me a link where I can get the chromium 65 version?

@jeremycjang all FF versions incl. the esr are crashing when I try to run them. Even when I run them manually by clicking the icon.

jeremycjang commented 5 years ago

@JoeFertig @dontic provided the steps for Chromium 65 here: https://github.com/timgrossmann/InstaPy/issues/4441#issuecomment-494371736

For convenience, here it is again:

Uninstall chromium 72: sudo apt-get purge chromium-browser rm -rf ~/.config/chromium

Install chromium 65 (because of #4033 (comment)): wget http://launchpadlibrarian.net/361669485/chromium-browser_65.0.3325.181-0ubuntu0.14.04.1_armhf.deb; sudo dpkg -i chromium-browser_65.0.3325.181-0ubuntu0.14.04.1_armhf.deb

Install codecs: wget http://launchpadlibrarian.net/361689926/chromium-codecs-ffmpeg_65.0.3325.181-0ubuntu0.16.04.1_armhf.deb; sudo dpkg -i chromium-codecs-ffmpeg_65.0.3325.181-0ubuntu0.16.04.1_armhf.deb

Use this with chromedriver 2.36 (install it like in the RPi tutorial)

unrelated to title issue, but inquired about in chain

Firefox-esr almost completed the full run for me, then crashed with the following stack trace:

  File "/home/pi/.local/lib/python3.5/site-packages/instapy/instapy.py", line 1726, in like_by_tags
    self.logger)
  File "/home/pi/.local/lib/python3.5/site-packages/instapy/like_util.py", line 236, in get_links_for_tag
    top_elements = browser.find_element_by_xpath('//main/article/div[1]')
  File "/home/pi/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 394, in find_element_by_xpath
    return self.find_element(by=By.XPATH, value=xpath)
  File "/home/pi/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 978, in find_element
    'value': value})['value']
  File "/home/pi/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/pi/.local/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response

   raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Failed to decode response from marionette

However, I believe could be because I ran a pretty long session liking 1500 posts. I'm fairly confident the program will work well using shorter sessions and am about to test this!

jeremycjang commented 5 years ago

@JoeFertig Firefox-esr crashed again with the same stack trace, appears to be a random occurrence

Here's to hoping the chromedriver fix resolves all these issues

dontic commented 5 years ago

It´s been 3 days now and none of my 2 instances of liking by tags bot have crashed using chromium 65 and chromedriver 2.36

Hopefully this means that the issue is solvable with this combination.

If I experience any issues I´ll write updates in this thread.

JoeFertig commented 5 years ago

So to close this issue:

sudo apt install chromium-browser chromium-codecs-ffmpeg

does the job. Anyone of you still experiencing problems?

dontic commented 5 years ago

So to close this issue:

sudo apt install chromium-browser chromium-codecs-ffmpeg

does the job. Anyone of you still experiencing problems?

Mine crashed again once, I think this solves the issue partially, as it crashes with much less frequency, taking days to do so.

Hopefully somebody finds another way to permanently solve this.

Joker6888 commented 5 years ago

Mine crashed again too, after one full day of running. Any ideas to solve the problem?

Joker6888 commented 5 years ago

Does anyone have an idea what the difference between the normal codecs and the -extra codecs is? my raspberry is giving an unexpected problem with the codec because the do not match with chromium while in upgrade-mode. Can I ignore the chromium and codec package in apt-get?

jeremycjang commented 5 years ago

I haven't experienced any more crashes using Chromium with the suggested solution!

I was observing the crashes occur when running a single program with a lot of activities for a long period of time, specifically when using the liking functions (never saw a crash with the follow actions). So I broke my single program down into smaller programs to run consecutively using simple Linux chaining, haven't experienced any issues other than the Pi generally overheating/rebooting once

InstaPy / InstaPy