Open RaviChan opened 4 years ago
Anyone met this issue?
DevTools listening on ws://127.0.0.1:59142/devtools/browser/5a32997f-1efb-4243-a270-6d1261dc5ec5
Traceback (most recent call last):
File "crawler.py", line 83, in
(Session info: headless chrome=77.0.3865.90)
Hi guys, I have a similar problem, the API cannot click properly on a point and can't fetch the data for posts_full option. Is it because of a change on the instagram website or is it my computer ?
Hi. Same error for me when I tried to get the number of likes for a post with a video.
I modified the function fetch_likes_plays in fetch.py file by changing browser.find_one(".QhbhU").click() at line 77 to element = browser.find_one(".QhbhU") browser.execute_script(element)
Also added def execute_script(self, t): self.driver.execute_script("arguments[0].click();", t) in browser.py file.
And it works for me. Maybe you can modify the code in a similar way. Hope it helps.
result.json.zip (https://github.com/huaying/instagram-crawler/files/3681900/result.json.zip) Thanks very much for your work, and this API is definitely the best Instagram crawler I can found on the internet. But when I try the example, I have found an interesting bug.
I have installed the toolkit with "chromedriver_mac64.zip" (chromedriver 77.0.3865.40)
Step1: % python3 crawler.py posts_full -u elenalinnn --fetch_comments Step2: we can see that 19/79 is ok, but the 20th one will show the following in red:
Failed to fetch the post: https://www.instagram.com/p/BxxzM7ACyaX/Traceback (most recent call last): File "/Users/xingchen/Documents/Python/spider-Ins/instagram-crawler-master/inscrawler/crawler.py", line 217, in _get_posts_full fetch_comments(browser, dict_post) File "/Users/xingchen/Documents/Python/spider-Ins/instagram-crawler-master/inscrawler/fetch.py", line 145, in fetch_comments show_comment_btn.click() File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click self._execute(Command.CLICK_ELEMENT) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/webelement.py", line 633, in _execute return self._parent.execute(command, params) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute self.error_handler.check_response(response) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable (Session info: headless chrome=77.0.3865.90)
After the running, we can get the 20th item as: {"key": "https://www.instagram.com/p/BrjMfkVFaSE/", "datetime": "2018-12-19T01:08:47.000Z"}, which is very different from others.
This bug will happen in item 20, 31, 48, 50, 55, 59, 60.
I can not find out what lead to this bug. Can you figure out this issue?
Thank you very much. Have a nice day.
Best Regards.