kevinzg / facebook-scraper

Scrape Facebook public pages without an API key
MIT License
2.47k stars 635 forks source link

get_page_info is giving error #444

Open tasnimaziz opened 3 years ago

tasnimaziz commented 3 years ago

i am scraping https://www.facebook.com/gopalitaliaofficial this page followers count with

scraper=FacebookScraper()
scraper.get_page_info('gopalitaliaofficial')

i am getting following error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
c:\users\tasnim\appdata\local\programs\python\python38\lib\site-packages\facebook_scraper\facebook_scraper.py in get_page_info(self, page, **kwargs)
    321             desc = resp.html.find("meta[name='description']", first=True)
--> 322             result["about"] = resp.html.find('#pages_msite_body_contents', first=True).text
    323         except Exception as e:

AttributeError: 'NoneType' object has no attribute 'text'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
<ipython-input-19-99d337d83883> in <module>
----> 1 scraper.get_page_info('gopalitaliaofficial')

c:\users\tasnim\appdata\local\programs\python\python38\lib\site-packages\facebook_scraper\facebook_scraper.py in get_page_info(self, page, **kwargs)
    326             resp = self.get(url)
    327             desc = resp.html.find("meta[name='description']", first=True)
--> 328             result["about"] = resp.html.find('#pages_msite_body_contents>div>div:nth-child(2)', first=True).text
    329             logger.error(e)
    330         if desc:

AttributeError: 'NoneType' object has no attribute 'text'
neon-ninja commented 3 years ago

This page looks more like a profile than a page. I note that the code:

pprint(get_profile('gopalitaliaofficial', cookies="cookies.txt"))

outputs:

{'Basic Info': 'Birthday',
 'Category': 'Public Figure',
 'Contact Info': {'Facebook': '/gopalitaliaofficial',
                  'Instagram': 'gopal_italia',
                  'Twitter': 'https://twitter.com/gopal_italia',
                  'Websites': ['https://www.bolegujarat.in/']},
 'Life Events': '',
 'Name': 'Gopal Italia',
 'Page Transparency\nSee All': 'See All\n'
                               'Facebook is showing information to help you '
                               'understand the purpose of this Page.\n'
                               'June 19, 2017\n'
                               'Page Creation Date\n'
                               'Page Admin Info\n'
                               'This Page can have multiple admins. They may '
                               'have permission to post content, comment or '
                               'send messages as the Page.\n'
                               'This Page is not currently running ads.',
 'cover_photo': 'https://scontent.fakl8-1.fna.fbcdn.net/v/t1.6435-9/fr/cp0/e15/q65/131037499_228429058639538_1988565575914376035_n.jpg?_nc_cat=101&ccb=1-5&_nc_sid=dd9801&efg=eyJpIjoidCJ9&_nc_ohc=lQr5RDL7hUcAX8Mc0NC&_nc_ht=scontent.fakl8-1.fna&oh=cffece1b49b7fe193cceef9377d7b72d&oe=61436571&manual_redirect=1',
 'cover_photo_text': "Cover Photo: Gopal Italia's photo.",
 'id': '100044173372516',
 'profile_picture': 'https://scontent.fakl8-1.fna.fbcdn.net/v/t1.6435-9/fr/cp0/e15/q65/199406670_341185664030543_3374874624252390383_n.jpg?_nc_cat=106&ccb=1-5&_nc_sid=85a577&efg=eyJpIjoidCJ9&_nc_ohc=CKCa4qqZi1AAX8CU7d-&_nc_oc=AQmQrSHCx5miF5ybiGJB4pL2EG3lBzRq4q3DNoI6gaQHIil7epC-XZZ3ZzIHNlzCepA&_nc_ht=scontent.fakl8-1.fna&oh=e9a795d2d96544794ed65638a8a43648&oe=61464240&manual_redirect=1'}

https://github.com/kevinzg/facebook-scraper/commit/1eceaaba14f1290e3db9b31c5abc9dec1b9b0def should fix the AttributeError. With this commit, the code:

pprint(get_page_info('gopalitaliaofficial', cookies="cookies.txt"))

outputs:

{'followers': 313008,
 'image': 'https://scontent.fakl8-1.fna.fbcdn.net/v/t1.6435-1/fr/cp0/e15/q65/199406670_341185664030543_3374874624252390383_n.jpg?_nc_cat=106&ccb=1-5&_nc_sid=c1fdac&efg=eyJpIjoidCJ9&_nc_ohc=CKCa4qqZi1AAX8CU7d-&_nc_oc=AQmQrSHCx5miF5ybiGJB4pL2EG3lBzRq4q3DNoI6gaQHIil7epC-XZZ3ZzIHNlzCepA&_nc_ht=scontent.fakl8-1.fna&oh=53303a29523e2051b08b39ad54935808&oe=6143502E',
 'name': 'Gopal Italia',
 'type': 'Organization',
 'url': '/gopalitaliaofficial'}