rocs-org / data-pipelines

Data engineering monorepo with Airflow, dbt and postgres
https://ornt.biologie.hu-berlin.de/airflow
4 stars 1 forks source link
airflow dbt postgres python

data-pipelines

tests Publish Docker image

Monorepo that contains data processing related libraries and services developed at ROCS

Data model documentation is at rocs-dbt-docs.netlify.app

This project uses poetry for package management and git-secret to manage secrets.

Contribute:

To contribute to this project please do the following:

  1. create a feature branch: git checkout -b feature/YOUR-FEATURE-NAME,
  2. develop your feature while making sure that tests and linting are still passing,
  3. To add a library or service enter the libraries or services folder and run $poetry new name_of_the_new_lib_or_service
  4. push your feature branch to the repo git push -u origin feature/YOUR-FEATURE-NAME and open a pull request.
  5. Ask one of the maintainers @davhin and @jakobkolb to review.