Open YovelD opened 2 years ago
With logging enabled :
Requesting page from: /trumilou/about/
Requesting page from: /trumilou/about/
Requesting page from: /trumilou/about/
Requesting page from: /trumilou/about/
Requesting page from: /trumilou/
Requesting page from: /trumilou/
Requesting page from: /trumilou/
Requesting page from: /trumilou/
No ld+json element
No ld+json element
No ld+json element
No ld+json element
Requesting page from: /trumilou/community
Requesting page from: /trumilou/community
Requesting page from: /trumilou/community
Requesting page from: /trumilou/community
Exception while requesting URL: https://m.facebook.com/trumilou/community
Exception: HTTPError('404 Client Error: Not Found for url: https://m.facebook.com/trumilou/community?locale=en_US')
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\site-packages\facebook_scraper\facebook_scraper.py", line 644, in get_page_info
ld_json = resp.html.find("script[type='application/ld+json']", first=True).text
AttributeError: 'NoneType' object has no attribute 'text'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\site-packages\facebook_scraper\facebook_scraper.py", line 852, in get
response.raise_for_status()
File "C:\Program Files\Python310\lib\site-packages\requests\models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://m.facebook.com/trumilou/community?locale=en_US
Exception while requesting URL: https://m.facebook.com/trumilou/community
Exception: HTTPError('404 Client Error: Not Found for url: https://m.facebook.com/trumilou/community?locale=en_US')
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\site-packages\facebook_scraper\facebook_scraper.py", line 644, in get_page_info
ld_json = resp.html.find("script[type='application/ld+json']", first=True).text
AttributeError: 'NoneType' object has no attribute 'text'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\site-packages\facebook_scraper\facebook_scraper.py", line 852, in get
response.raise_for_status()
File "C:\Program Files\Python310\lib\site-packages\requests\models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://m.facebook.com/trumilou/community?locale=en_US
Exception while requesting URL: https://m.facebook.com/trumilou/community
Exception: HTTPError('404 Client Error: Not Found for url: https://m.facebook.com/trumilou/community?locale=en_US')
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\site-packages\facebook_scraper\facebook_scraper.py", line 644, in get_page_info
ld_json = resp.html.find("script[type='application/ld+json']", first=True).text
AttributeError: 'NoneType' object has no attribute 'text'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\site-packages\facebook_scraper\facebook_scraper.py", line 852, in get
response.raise_for_status()
File "C:\Program Files\Python310\lib\site-packages\requests\models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://m.facebook.com/trumilou/community?locale=en_US
Exception while requesting URL: https://m.facebook.com/trumilou/community
Exception: HTTPError('404 Client Error: Not Found for url: https://m.facebook.com/trumilou/community?locale=en_US')
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\site-packages\facebook_scraper\facebook_scraper.py", line 644, in get_page_info
ld_json = resp.html.find("script[type='application/ld+json']", first=True).text
AttributeError: 'NoneType' object has no attribute 'text'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\site-packages\facebook_scraper\facebook_scraper.py", line 852, in get
response.raise_for_status()
File "C:\Program Files\Python310\lib\site-packages\requests\models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://m.facebook.com/trumilou/community?locale=en_US
'NoneType' object has no attribute 'text'
'NoneType' object has no attribute 'text'
'NoneType' object has no attribute 'text'
'NoneType' object has no attribute 'text'
Requesting page from: trumilou
Requesting page from: trumilou
Requesting page from: trumilou
Requesting page from: trumilou
Got exact timestamp from publish_time: 2022-08-07 11:28:43
Got exact timestamp from publish_time: 2022-08-07 11:28:43
Got exact timestamp from publish_time: 2022-08-07 11:28:43
Got exact timestamp from publish_time: 2022-08-07 11:28:43
Fetching https://m.facebook.com/photo.php?fbid=471115638353044&id=100063638347002&set=a.195694489228495&source=48&refid=17
Fetching https://m.facebook.com/photo.php?fbid=471115638353044&id=100063638347002&set=a.195694489228495&source=48&refid=17
Fetching https://m.facebook.com/photo.php?fbid=471115638353044&id=100063638347002&set=a.195694489228495&source=48&refid=17
Fetching https://m.facebook.com/photo.php?fbid=471115638353044&id=100063638347002&set=a.195694489228495&source=48&refid=17
Fetching https://m.facebook.com/photo/view_full_size/?fbid=471115638353044&ref_component=mbasic_photo_permalink&ref_page=%2Fwap%2Fphoto.php&refid=13
Fetching https://m.facebook.com/photo/view_full_size/?fbid=471115638353044&ref_component=mbasic_photo_permalink&ref_page=%2Fwap%2Fphoto.php&refid=13
Fetching https://m.facebook.com/photo/view_full_size/?fbid=471115638353044&ref_component=mbasic_photo_permalink&ref_page=%2Fwap%2Fphoto.php&refid=13
Fetching https://m.facebook.com/photo/view_full_size/?fbid=471115638353044&ref_component=mbasic_photo_permalink&ref_page=%2Fwap%2Fphoto.php&refid=13
[471115671686374] Extract method extract_video didn't return anything
[471115671686374] Extract method extract_video didn't return anything
[471115671686374] Extract method extract_video didn't return anything
[471115671686374] Extract method extract_video didn't return anything
[471115671686374] Extract method extract_video_thumbnail didn't return anything
[471115671686374] Extract method extract_video_thumbnail didn't return anything
[471115671686374] Extract method extract_video_thumbnail didn't return anything
[471115671686374] Extract method extract_video_thumbnail didn't return anything
[471115671686374] Extract method extract_video_id didn't return anything
[471115671686374] Extract method extract_video_id didn't return anything
[471115671686374] Extract method extract_video_id didn't return anything
[471115671686374] Extract method extract_video_id didn't return anything
Fetching 471115671686374
Fetching 471115671686374
Fetching 471115671686374
Fetching 471115671686374
[471115671686374] Extract method extract_video_meta didn't return anything
[471115671686374] Extract method extract_video_meta didn't return anything
[471115671686374] Extract method extract_video_meta didn't return anything
[471115671686374] Extract method extract_video_meta didn't return anything
[471115671686374] Extract method extract_factcheck didn't return anything
[471115671686374] Extract method extract_factcheck didn't return anything
[471115671686374] Extract method extract_factcheck didn't return anything
[471115671686374] Extract method extract_factcheck didn't return anything
[471115671686374] Extract method extract_share_information didn't return anything
[471115671686374] Extract method extract_share_information didn't return anything
[471115671686374] Extract method extract_share_information didn't return anything
[471115671686374] Extract method extract_share_information didn't return anything
[471115671686374] Extract method extract_listing didn't return anything
[471115671686374] Extract method extract_listing didn't return anything
[471115671686374] Extract method extract_listing didn't return anything
[471115671686374] Extract method extract_listing didn't return anything
Friend_count extraction failed: list index out of range
Friend_count extraction failed: list index out of range
Friend_count extraction failed: list index out of range
Friend_count extraction failed: list index out of range
Follower_count extraction failed: 'NoneType' object has no attribute 'text'
Follower_count extraction failed: 'NoneType' object has no attribute 'text'
Follower_count extraction failed: 'NoneType' object has no attribute 'text'
Follower_count extraction failed: 'NoneType' object has no attribute 'text'
Fetching /trumilou?v=following
Fetching /trumilou?v=following
Fetching /trumilou?v=following
Fetching /trumilou?v=following
Fetching https://m.facebook.com/photo/view_full_size/?fbid=196728852458392&ref_component=mbasic_photo_permalink&ref_page=%2Fwap%2Fphoto.php&refid=13
Fetching https://m.facebook.com/photo/view_full_size/?fbid=196728852458392&ref_component=mbasic_photo_permalink&ref_page=%2Fwap%2Fphoto.php&refid=13
Fetching https://m.facebook.com/photo/view_full_size/?fbid=196728852458392&ref_component=mbasic_photo_permalink&ref_page=%2Fwap%2Fphoto.php&refid=13
Fetching https://m.facebook.com/photo/view_full_size/?fbid=196728852458392&ref_component=mbasic_photo_permalink&ref_page=%2Fwap%2Fphoto.php&refid=13
Fetching https://m.facebook.com/photo/view_full_size/?fbid=195694505895160&ref_component=mbasic_photo_permalink&ref_page=%2Fwap%2Fphoto.php&refid=13
Fetching https://m.facebook.com/photo/view_full_size/?fbid=195694505895160&ref_component=mbasic_photo_permalink&ref_page=%2Fwap%2Fphoto.php&refid=13
Fetching https://m.facebook.com/photo/view_full_size/?fbid=195694505895160&ref_component=mbasic_photo_permalink&ref_page=%2Fwap%2Fphoto.php&refid=13
Fetching https://m.facebook.com/photo/view_full_size/?fbid=195694505895160&ref_component=mbasic_photo_permalink&ref_page=%2Fwap%2Fphoto.php&refid=13
Requesting page from: https://m.facebook.com/trumilou/about/
Requesting page from: https://m.facebook.com/trumilou/about/
Requesting page from: https://m.facebook.com/trumilou/about/
Requesting page from: https://m.facebook.com/trumilou/about/
Try pass the reviews=True argument to get_page_info. Alternatively pass reviews="generator" to get a generator back. Alternatively call _scraper.get_page_reviews to bypass get_page_info if you only need reviews.
Thank you for your answer. I still can't seem to be able to get reviews, with either reviews=True or reviews="generator". I'm not sure how to use _scraper.get_page_reviews.
from facebook_scraper import *
from pprint import pprint
import tqdm as notebook_tqdm
set_cookies("facebook.com_cookies.txt")
set_user_agent("Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)")
for review in get_page_info("trumilou", reviews=True)['reviews']:
pprint(review)
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\site-packages\facebook_scraper\utils.py", line 279, in safe_consume
for item in generator:
File "C:\Program Files\Python310\lib\site-packages\facebook_scraper\facebook_scraper.py", line 555, in get_page_reviews
response = self.get(more_url)
File "C:\Program Files\Python310\lib\site-packages\facebook_scraper\facebook_scraper.py", line 852, in get
response.raise_for_status()
File "C:\Program Files\Python310\lib\site-packages\requests\models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://m.facebook.com/trumilou/reviews?locale=en_US
https://m.facebook.com/trumilou/reviews?locale=en_US doesn't work in my browser either, this page must have reviews disabled
Non mobile version works : https://www.facebook.com/Trumilou/reviews/?locale=en_US, which is weird. Do you know what could be happening ? I can't get any mobile page to work with reviews.
Hi everyone,
I tried to retrieve reviews from a page but am getting errors.
The error I get :
When I try pprint(get_page_info("trumilou")), I don't get info about reviews either :
I'm using Facebook Scraper 0.2.58. This happens no matter the page I try.