Open cyri113 opened 1 year ago
The Web3 Foundation has contracted TogetherCrew to deliver a Twitter Dashboard for monitoring community health.
/auth/twitter
/twitter/:handle
/twitter/:handle/refresh
x hours
sagaId
/saga/:sagaId
TWITTER_REFRESH
QUEUE.TWITTER_BOT
Event.TWITTER_BOT.EXTRACT
QUEUE.TWITTER_ANALYZER
Event.TWITTER_ANALYZER.RUN
Event.TWITTER_BOT.SEND_MESSAGE
sequenceDiagram title Twitter Integration actor user participant frontend participant api participant rabbit-mq participant twitter-bot-server participant twitter-bot-redis participant twitter-bot-worker participant twitter-analyzer-server participant twitter-analyzer-redis participant twitter-analyzer-worker participant twitter user -->> frontend: Goes to /twitter frontend -->>+api: Authenticate using Twitter api -->>+twitter: Authenticate twitter -->>-api: Response api -->>-frontend: Authenticated frontend -->> frontend: Redirect to /twitter/:handle frontend -->>+api: GET /twitter/:handle/refresh api -->> rabbit-mq: Publish(Queue.TWITTER_BOT, Event.TWITTER_BOT.EXTRACT) api -->>- frontend: Response (SagaId) frontend --> api: Subscribe to socket /saga/:sagaId rabbit-mq -->> twitter-bot-server: Subscribe(Queue.TWITTER_BOT, Event.TWITTER_BOT.EXTRACT) twitter-bot-server -->> twitter-bot-redis: Enqueue Job twitter-bot-redis -->>+twitter-bot-worker: Dequeue Job twitter-bot-worker -->> twitter-bot-worker: Run extraction script twitter-bot-worker -->>-rabbit-mq: Publish(Queue.TWITTER_ANALYZER, Event.TWITTER_ANALYZER.RUN) rabbit-mq -->> twitter-analyzer-server: Subscribe(Queue.TWITTER_ANALYZER, Event.TWITTER_ANALYZER.RUN) twitter-analyzer-server -->> twitter-analyzer-redis: Enqueue Job twitter-analyzer-redis -->>+twitter-analyzer-worker: Dequeue Job twitter-analyzer-worker -->> twitter-analyzer-worker: Run analytics script twitter-analyzer-worker -->>-rabbit-mq: Publish(Queue.TWITTER_BOT, Event.TWITTER_BOT.SEND_MESSAGE) rabbit-mq -->> twitter-bot-server: Subscribe(Queue.TWITTER_BOT, Event.TWITTER_BOT.SEND_MESSAGE) twitter-bot-server -->> twitter-bot-redis: Enqueue Job twitter-bot-redis -->> twitter-bot-worker: Dequeue Job twitter-bot-worker -->>+ twitter: Send message twitter -->>- twitter-bot-worker: Response frontend -->>+ api: GET /twitter/:handle/followers api -->>- frontend: Response frontend -->>+ api: GET /twitter/:handle/posts api -->>- frontend: Response frontend -->>+ api: GET /twitter/:handle/metrics api -->>- frontend: Response
twitter/:handle/refresh
Queue.TWITTER_BOT
twitter/:handle/metrics/followers
twitter/:handle/metrics/posts
Queue.TWITTER_ANALYZER
based on python-service
We've defined a base event for bots which is FETCH. Isn't it better to use the term FETCH instead of EXTRACT for TWITTER_BOT?
FETCH
EXTRACT
TWITTER_BOT
Overview
The Web3 Foundation has contracted TogetherCrew to deliver a Twitter Dashboard for monitoring community health.
High Level Solution
/auth/twitter
/twitter/:handle
/twitter/:handle/refresh
x hours
- show datax hours
- show data + propose refresh (button)/twitter/:handle/refresh
should create a saga, initiate the choreography and respond with asagaId
/saga/:sagaId
/twitter/:handle/refresh
)TWITTER_REFRESH
choreography with transactions:QUEUE.TWITTER_BOT
, event:Event.TWITTER_BOT.EXTRACT
}QUEUE.TWITTER_ANALYZER
, event:Event.TWITTER_ANALYZER.RUN
}QUEUE.TWITTER_BOT
, event:Event.TWITTER_BOT.SEND_MESSAGE
}Workflow
By Service
api
/twitter/:handle
upon successful authenticationtwitter/:handle/refresh
Event.TWITTER_BOT.EXTRACT
onQueue.TWITTER_BOT
twitter/:handle/metrics/followers
(tbd)twitter/:handle/metrics/posts
(tbd)/saga/:sagaId
broker libraries (py + ts)
TWITTER_REFRESH
choreographyQueue.TWITTER_BOT
queueQueue.TWITTER_ANALYZER
queueEvent.TWITTER_BOT.EXTRACT
eventEvent.TWITTER_BOT.SEND_MESSAGE
eventEvent.TWITTER_ANALYZER.RUN
eventtwitter-bot
based on python-service
server
Event.TWITTER_BOT.EXTRACT
onQueue.TWITTER_BOT
worker
Event.TWITTER_ANALYZER.RUN
onQUEUE.TWITTER_ANALYZER
twitter-analyzer
based on python-service
server
Event.TWITTER_ANALYZER.RUN
onQueue.TWITTER_ANALYZER
worker
Event.TWITTER_BOT.SEND_MESSAGE
onQueue.TWITTER_BOT
frontend
/auth/twitter
endpoint (authentication)/twitter/:handle
endpoint (dashboard)/saga/:sagaId