Error on scraping followers

[Thicool]

Works fine but scraping followers gives me:

Scraping followers... Traceback (most recent call last): File "instagramcrawler.py", line 302, in caption='None') File "instagramcrawler.py", line 96, in crawl self.scrape_followers_or_following(crawl_type, query, number) File "instagramcrawler.py", line 214, in scrape_followers_or_following title = self._driver.find_element_by_xpath(FOLLOW_PATH) File "...\selenium\webdriver\remote\webdriver.py", line 313, in find_element_by_xpath return self.find_element(by=By.XPATH, value=xpath) File "...\selenium\webdriver\remote\webdriver.py", line 791, in find_element 'value': value})['value'] File "...\selenium\webdriver\remote\webdriver.py", line 256, in execute self.error_handler.check_response(response) File "...\selenium\webdriver\remote\errorhandler.py", line 194, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: //div[contains(text(), 'Followers')]

Any idea? Thanks for your help.

[tzuhsial]

Can you give me more information & environment on your query? (E.g. the command you used to crawl followers) Thanks!

[Thicool]

Hey Antonie,

thanks for your reply. Here are the commands that i use and the response i get. I have no problem scraping pictures. After executing the code, firefox starts and i enter my login (in case of followers), which then opens the instagram-account page and open the follower-list. However, script stops at that. I hope you can help me! Here is the command line in and output:

---

C:\Users\Jankl\PycharmProjects\InstagramCrawler-master>python instagramcrawler.py -q instagram -t followers -n 30 dir_prefix: ./data/, query: instagram, crawl_type: followers, number: 30, caption: False You will need to login to crawl followers

Scraping followers... Traceback (most recent call last): File "instagramcrawler.py", line 297, in main() File "instagramcrawler.py", line 293, in main caption=args.caption) File "instagramcrawler.py", line 96, in crawl self.scrape_followers_or_following(crawl_type, query, number) File "instagramcrawler.py", line 214, in scrape_followers_or_following title = self._driver.find_element_by_xpath(FOLLOW_PATH) File "C:\Users\Jankl\Anaconda2\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 313, in find_element_by_xpath return self.find_element(by=By.XPATH, value=xpath) File "C:\Users\Jankl\Anaconda2\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 791, in find_element 'value': value})['value'] File "C:\Users\Jankl\Anaconda2\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 256, in execute self.error_handler.check_response(response) File "C:\Users\Jankl\Anaconda2\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 194, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: //div[contains(text(), 'Followers')]

---

When i add paranteces on followers, i get another error message:

---

C:\Users\Jankl\PycharmProjects\InstagramCrawler-master>python instagramcrawler.py -q instagram -t 'followers' -n 30 dir_prefix: ./data/, query: instagram, crawl_type: 'followers', number: 30, caption: False Saving... Saving to directory..../data/instagram Quitting driver...

C:\Users\Jankl\PycharmProjects\InstagramCrawler-master>

---

Thanks for your help :)

2017-06-30 14:22 GMT+02:00 Antonie Lin notifications@github.com:

Can you give me more information & environment on your query? (E.g. the command you used to crawl followers) Thanks!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/iammrhelo/InstagramCrawler/issues/7#issuecomment-312253236, or mute the thread https://github.com/notifications/unsubscribe-auth/AVm_4jEV85BqAgbrUue8mIGbsaaeX9n1ks5sJOiUgaJpZM4OJSl2 .

[tzuhsial]

@Thicool Did you login your instagram account? (The webdriver will open up the browser for you to login)

[Thicool]

Yes it opens firefox and i login, then the driver continues to visit the follower Page. Everything works fine until here. Maybe i try another computer later.

Regards, Jan

Am 03.07.2017 6:11 vorm. schrieb "Antonie Lin" notifications@github.com:

Did you login your instagram account? (The webdriver will open up the browser for you to login)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/iammrhelo/InstagramCrawler/issues/7#issuecomment-312545648, or mute the thread https://github.com/notifications/unsubscribe-auth/AVm_4ufCdBxcOeHSb2AihsgLXtdfjRBJks5sKGnUgaJpZM4OJSl2 .

[tzuhsial]

@Thicool I think its because after clicking on the element, my code only waits for 1 second before scrolling the followers list I'll fix and push it later

Antonie

[tzuhsial]

@Thicool Pushed, please check if you have time... And also, thanks for notifying!

Regards, Antonie Lin

[Thicool]

Thanks for your work, but the problem still exists in my case. I Attached Error Message and Gecklog.

Hope that helps!

error.txt geckodriver.txt

[tzuhsial]

@Thicool It works for me when crawling followers/ following at the moment The only solution I know have is to set the timeout value a bit larger(make it wait a little longer)...

[mrpatricque]

Hi there! I am encountering the same issue here! How can I set the timeout value a bit larger? I can crawl for photos, but get the same error for followers/following. When crawling photos, there is no login done, and when I include crawling reactions, I get a different, more complicated error. Thanks!