use tweepy api when twitter.com url is detected as source

soprasteria / cybersecurity-dfm

Data Feed Manager (news watch orchestrator to predict topic with deepdetect and store cleaned text in elasticsearch)

GNU General Public License v3.0

40 stars 14 forks source link

use tweepy api when twitter.com url is detected as source #5

Closed acabrol closed 6 years ago

acabrol commented 6 years ago

When an url is submitted from twitter.com it is not yet recognized as a tweet so the tweepy api is not used and link included are not extracted.

During source adress crawling we could detect is the domain is twitter.com and then use tweepy to extract links in the tweet to process source news instead of processing the tweet message only.

acabrol commented 6 years ago

Fixed when "twitter.com" is found in the link use Tweepy instead of link parser....