Feat add hivemind etl scripts - Githubissues

TogetherCrew / airflow-dags

1 stars 1 forks source link

Feat add hivemind etl scripts #15

Closed amindadgar closed 10 months ago

amindadgar commented 11 months ago

In this PR we're aiming to add

Discord vectorstore sctipts
Discord summary vectorstore scripts
Discourse vectorstore sctripts
Gdrive vectorstore scripts

Notes:

We've finished the implementation and testing for the two first items.
For item 3, we still need to check with a newer version of staging data (the staging data needed some updates)
Item 4, is work-in-progress and we would complete it in another PR.