reworking how total post count is pulled from page; fixes issue #26

wwwehr commented 6 years ago

instagram has changed the format of the page; it no longer has a CSS_LOAD_MORE that I can detect so I removed that as well. I can now use this command successfully:

$ python instagramcrawler.py -q '#breakfast' -n 5
dir_prefix: ./data/, query: #breakfast, crawl_type: photos, number: 5, caption: False, authentication: None
posts: 70478766, number: 5
Scraping photo links...
Number of photo_links: 33
Saving...
Downloading 5 images to ata/breakfast.hashtag
Quitting driver...

using geckodriver 0.20.0 for osx:

https://github.com/mozilla/geckodriver/releases/download/v0.20.0/geckodriver-v0.20.0-macos.tar.gz

JoaquinMontesinos commented 6 years ago

for me intagramcrawler.py doesn´t work. Error in line 117 self.scroll_to_num_of_posts(number)

vuongducdai commented 5 years ago

This works only the number of photos you want to crawl is less than the number of photos in the current page, it cannot automatically scroll down for us. In other words, this code can only crawl in one page only. Any idea?

Cupido10 commented 5 years ago

I got the same problem here, Did you guys get any solutions?

tzuhsial / InstagramCrawler

reworking how total post count is pulled from page; fixes issue #26 #27