search

DiamondDAO / discord-etl

This program takes the output from `discord-bot` then transforms and loads into a database
0 stars 1 forks source link

Include Threads in Datamodel #3

Open nathankoerschner opened 2 years ago

nathankoerschner commented 2 years ago

Discord threads seem to be important to distinguish. Currently, our bot parses them and stores them as channels.

In order to make thread data easy to query, they should be included in our datamodel.

To stay consistent with the rest of the model, these will have an entity table and a history table.

nathankoerschner commented 2 years ago

Determined that the best way of doing this is to group threads under channels. Since channels have the type attribute, it will be easy to parse the channels table for all threads.

Likewise, thread histories (thread delete and thread update events) will be sent to the channel histories table.

nathankoerschner commented 2 years ago

Included thread related events in the bot, now all of the requisite data to query thread info is sent to s3.

Since the underlying datamodel didn't change, there's no need to make any changes to this repo (discord-etl)

nathankoerschner commented 2 years ago

@clemp This issue is ready for review.