search

brutalsavage / facebook-post-scraper

Facebook Post Scraper 🕵️🖱️
GNU General Public License v3.0
328 stars 118 forks source link

chrom path #12

Closed daniel41wq closed 4 years ago

daniel41wq commented 4 years ago

hello, hope you are good, when I'm trying to run the script on windows it keeps giving me this error

Traceback (most recent call last): File "scraper.py", line 317, in postBigDict = extract(page=args.page, numOfPost=args.len, infinite_scroll=infinite, scrape_comment=scrape_comment) TypeError: extract() missing 1 required positional argument: 'chromedriver_path'

and the chrome driver is in the same directory

brutalsavage commented 4 years ago

Hi, sorry about that. One of the previous commits added the need to insert the path of the chromedriver. I removed that for now, if you reclone and run it should work. Let me know if you still have issues.

daniel41wq commented 4 years ago

is it possible to scrap just a few posts !

brutalsavage commented 4 years ago

you can set the -len argument to be a small number in the command line argument

daniel41wq commented 4 years ago

but it keeps scrolling down this is my command python scraper.py -page Nike -len 3 -usage PS -comments y

brutalsavage commented 4 years ago

It is because you enabled comments, the Nike page has a lot of comments so the scraper spends alot of time clicking on "view more comments" to try to get all the comments. For pages that are very popular like Nike, the scraper will take a long time even on a few posts with scraping comments enabled

daniel41wq commented 4 years ago

yes you are right and when i switched to another page this line returns an empty list I think it must be updated postComments = item.find_all(attrs={"aria-label": "Comment"})