RnDAO / tc-botComm

GNU General Public License v3.0

3 stars 4 forks source link

Proof of concept for the new architecture #70

Closed mehrdadmms closed 1 year ago

mehrdadmms commented 1 year ago

Goal

we need a way to make our data extraction and analysis scalable.

needs

Parallelly extract information from different sources and different guilds
Parallelly analyze the stored raw information
Have different analysis scripts working at the same time
Have the ability to add more nodes of same analysis script for parallel analysis and load balancing
Have the ability to run second, third, and Nth level analysis (analysis that uses the data of another analysis)

Tasks

[ ] Create an instance of RabbitMQ in dev environment
[ ] Review and analyze the architecture
[ ] Research on best practices
[ ] Finalize the architecture
[ ] Implement a basic version of the architecture as proof of concept

mehrdadmms commented 1 year ago

My suggestion for the architecture https://miro.com/app/board/uXjVPkg255E=/?share_link_id=678339601738