kevinzg / facebook-scraper

Scrape Facebook public pages without an API key
MIT License
2.45k stars 633 forks source link

Temporary Banned when getting reactions - Optimize code #986

Open doobybug opened 1 year ago

doobybug commented 1 year ago

I am trying to get yesterday's posts from 16 different portals. I want to get the reaction_count as well. Unfortunately I am getting temporarily banned every time I run the code. Is there a way I can optimize the code?

def get_yesterdays_fbposts(p):
    print("----------------------------------------------------------------------")
    print ("Yesterday")
    print("Scraping Portal: "+p)
    results = []
    start_url = None
    i = 0
    x= get_posts(p, page_limit=None, start_url=start_url, request_url_callback=handle_pagination_url, options={'allow_extra_requests': True, 'reactions': True, 'posts_per_page': 200})   
    for post in x:
        if(post['time']!=None):  
            if(post['time'].strftime('%Y%m%d')==yesterday.strftime('%Y%m%d')):
                post['Portal'] = p
                print(post['post_url'])
                results.append(post)
            elif(post['time'].strftime('%Y%m%d')>yesterday.strftime('%Y%m%d')):
                continue
            else:
                #pinned post
                if(i!=0):
                    break
        i=i+1
        #sleep for 10 sec 
        time.sleep(10)

    print(p+" Done") 
    return pd.DataFrame(results)
Roeland-Dubel commented 1 year ago

I have the exact same issue! I can scrape the text almost unlimited, but when I want the reactions as well (which I need), it bans me quite quickly (after like a 100 posts), regardless if I put a delay on it.

Alanjv05 commented 1 year ago

Same issue

leviethung2103 commented 1 year ago

Same issue. Has anyone the solution ?