kevinzg / facebook-scraper

Scrape Facebook public pages without an API key
MIT License
2.45k stars 633 forks source link

Using A Proxy #1058

Closed omarchouman closed 1 year ago

omarchouman commented 1 year ago

Can I use a proxy with this while using the get_posts() function?

If so how can I do that?

Snippet of my code for extra input:

import time
from facebook_scraper import *
import json

results = []
start_url = None

page_name = input("Enter the page Id: ")

def handle_pagination_url(url):
    global start_url
    start_url = url
set_cookies("cookies.txt")

post_count = 0

while True:
    try:
        for post in get_posts(page_name, page_limit=2, start_url=start_url, request_url_callback=handle_pagination_url):
            print(json.dumps(post['text']))
            post_count += 1
            if post_count == 2:
                break
        print("All done")
        break
    except exceptions.TemporarilyBanned:
        print("Temporarily banned, sleeping for 10m")
        time.sleep(600)

Thanks Everyone!

chelishchev commented 1 year ago

You can just setup proxy by invoking facebook_scraper.set_proxy(proxy)

omarchouman commented 1 year ago

Oh Thanks, Didn't See That Function!