airflow-dags

In this repository, we've shared all dags for TogetherCrew. The dags are

• GitHub API data ETL DAG
• Hivemind discord ETL DAG
• Hivemind discourse ETL DAG
• Hivemind GitHub ETL DAG

Note: the Hivemind ETL dags, are related to the data processing of togethercrew's LLM.

Running the app

You can quickly launch the application using Docker Compose:

docker-compose -f docker-compose.yaml up

reference: Running Airflow in Docker