-
Basically according to Google there will be new crawler that uses
`Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safar…
-
When I start my react native app via npm start i got an Error
`jest-haste-map: Watchman crawl failed. Retrying once with node crawler.
Usually this happens when watchman isn't running. Create an e…
-
When crawling facebook.com the crawler goes into infinite crawling where /help keeps getting appended to the url. When manually browsing the link, it redirects to metacareers.com but not while crawlin…
-
These are the UA's of crawlers which are not detected by this library as spiders. I will continue report them here in this thread.
```
python-requests/2.18.4
Go-http-client/2.0
FeedViewer/1.0 …
-
So SEO was discussed in issue #101 for the Google crawler, but this is concerned with the Facebook and Twitter crawlers. They both do not interpret JavaScript, which makes it impossible to dynamically…
-
Hi, I was trying to use the code below.
`import facebook_crawler pageurl= "https://www.facebook.com/TaylorSwift"
data = facebook_crawler.Crawl_PagePosts(pageurl=pageurl, until_date='2023-01-01')
da…
-
### Have you read the Contributing Guidelines on issues?
- [X] I have read the [Contributing Guidelines on issues](https://github.com/facebook/docusaurus/blob/main/CONTRIBUTING.md#issues).
### Prere…
-
Hi there,
I'm wondering if "logged-in crawls" can be done using TrackerRadar. The crawler needs to preserve logged-in status to Facebook/Google.
-
Hello,
Sorry for bothering you again, compiled went great, after your commit, thanks.
Another problem, i think it has to do with utf-8 url chars now, i guess nothing important since its in test only…
-
I got a warning I think because my Cognito IDP URL results in a 3xx/400 without context, so the crawler thinks it's no good.
https://developers.facebook.com/community/threads/605080477083441/
> …