search

bisguzar / twitter-scraper

Scrape the Twitter Frontend API without authentication.
MIT License
3.91k stars 599 forks source link

Allow search by date ranges #136

Open jcea opened 4 years ago

jcea commented 4 years ago

As documented in #96, you can not access all the tweet account history simply scrolling. There is a limit of around ~800 tweets.

Nevertheless, the search URL allows to specify date ranges. For instance:

https://twitter.com/search?q=from%3Ajcea%20since%3A2015-01-01%20until%3A2015-02-01&src=typed_query&f=live

If we could parse those results...

icmpnorequest commented 4 years ago

@jcea Hi,

I tried to scrape tweets with the official provided url https://twitter.com/i/profiles/show/realDonaldTrump/timeline/tweets?include_available_features=1&include_entities=1&include_new_items_bar=true

However, there's no response and return 403 error code, as I mentioned in https://github.com/bisguzar/twitter-scraper/issues/135#issue-622458276. I was wondering if you met the same issue or could you please give me some guidance on it?

Thanks in advance.

jcea commented 4 years ago

I didn't dig the details, but projects like https://github.com/Mottl/GetOldTweets3/ allow to search by date range.

jcea commented 4 years ago

@icmpnorequest , sorry I can not help you.

Please, do not hijack github issues unrelated to your problem.

Thanks.