search

Etwas-Builders / Twitter-Source-Bot

Ever wanted to know the source of a tweet? Just @whosaidthis_bot and I'll tell you where it came from
https://twittersourcebot.tech/
15 stars 1 forks source link

t.co urls in keywords string gives no results from scraper #37

Closed bkataru closed 4 years ago

bkataru commented 4 years ago

Tweet: image

Link: https://twitter.com/nytpolitics/status/1259161485458235392

Keywords string: "Barack Obama Trump administration Friday address aides, response coronavirus pandemic “chaotic" DOJ’s decision drop charges Michael Flynn “rule law" https://t.co/vdkJ25DoAg attacked former criticizing its warning against endangered"

No results produced from google scraper. Search worked for the following keywords string:

Working keywords string: "Barack Obama Trump administration Friday address aides, response coronavirus pandemic “chaotic" DOJ’s decision drop charges Michael Flynn “rule law" attacked former criticizing its warning against endangered"

bkataru commented 4 years ago

The mention of t.co was not an issue but instead it was quotations ('"‘’“”) in the tweet body that gave no results (due to quotations being an important search technique in google). Used RegEx to replace quotations in the tweet body before searching