issues
search
rocs-org
/
data-pipelines
Data engineering monorepo with Airflow, dbt and postgres
https://ornt.biologie.hu-berlin.de/airflow
4
stars
1
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
add mysql target to dbt profiles
#229
davhin
closed
1 year ago
1
Feature/try mlflow
#228
davhin
opened
1 year ago
2
Fix/update gitignore
#227
davhin
closed
1 year ago
1
add types of epoch values for lookup
#226
davhin
closed
1 year ago
2
Refactor airflow dbt task to run dbt models as individual airflow tasks
#225
jakobkolb
closed
1 year ago
2
Increase observability in dbt model application
#224
jakobkolb
closed
1 year ago
0
Fix DB migrations
#223
jakobkolb
closed
1 year ago
1
Fix DB migrations
#222
jakobkolb
closed
1 year ago
0
Cleanup and set concurrency of epoch dag to 1.
#221
jakobkolb
closed
1 year ago
1
Reduce epoch dag concurrency to 2.
#220
jakobkolb
closed
1 year ago
1
Load of epoch dags crashes server
#219
jakobkolb
closed
1 year ago
0
Rename github actions.
#218
jakobkolb
closed
1 year ago
1
Fix CD pipeline.
#217
jakobkolb
closed
1 year ago
1
Enforce documentation, linting, and test existence of dbt models
#216
davhin
opened
2 years ago
1
Add clickhouse migrations cd
#215
jakobkolb
closed
2 years ago
1
Add CD for clickhouse migrations
#214
jakobkolb
closed
2 years ago
0
Sync epoch data
#213
jakobkolb
closed
2 years ago
1
Sync epoch data
#212
jakobkolb
closed
2 years ago
0
fix deprecation warnings from Airflow
#211
davhin
closed
2 years ago
1
Remove production secrets from airflow and add unencrypred test secrets.
#210
jakobkolb
closed
2 years ago
1
Remove airflow production secrets
#209
jakobkolb
closed
2 years ago
0
Sync epoch data with inhouse CH instance
#208
jakobkolb
closed
2 years ago
2
Fix CD pipelines
#207
jakobkolb
closed
2 years ago
1
Add onetime vitaldata update
#206
davhin
closed
2 years ago
1
Update dependencies.
#205
jakobkolb
closed
2 years ago
1
Change csv parsing with pandas.
#204
jakobkolb
closed
2 years ago
1
Update covid infections url according to rki github changes.
#203
jakobkolb
closed
2 years ago
1
Covid cases pipeline is broken
#202
jakobkolb
closed
2 years ago
0
Fix docker image upload.
#201
jakobkolb
closed
2 years ago
1
Fix publish docker action.
#200
jakobkolb
closed
2 years ago
1
Package dag code with docker image do deploy on kubernetes
#199
jakobkolb
closed
2 years ago
1
currently we cannot push results to any place outside
#198
davhin
opened
2 years ago
0
document vital data table for datenspende, add custom info on front page
#197
davhin
closed
2 years ago
3
add dbt models for user research into vital and survey donation behavior
#196
davhin
closed
1 year ago
7
Add test type to homogenized features.
#195
jakobkolb
opened
2 years ago
1
Add rolling window aggregates and derived vital features…
#194
jakobkolb
closed
2 years ago
1
Remove compiled sql.
#193
jakobkolb
closed
2 years ago
1
Deploy docs to Netlify
#192
jakobkolb
closed
2 years ago
5
Host dbt docs with pull request previews
#191
jakobkolb
closed
2 years ago
0
Migrate scripps post processing to dbt
#190
jakobkolb
closed
2 years ago
0
Change dag concurrency and worker pool size.
#189
jakobkolb
closed
2 years ago
0
Change dag concurrency and worker pool size.
#188
jakobkolb
closed
2 years ago
0
Allow parallel backfills.
#187
jakobkolb
closed
2 years ago
0
Change batching and worker pool size.
#186
jakobkolb
closed
2 years ago
0
Restrict loading vitals to necesssary dates.
#185
jakobkolb
closed
2 years ago
0
Change database handling in workers and decrease worker pool size.
#184
jakobkolb
closed
2 years ago
0
Optimize select queries.
#183
jakobkolb
closed
2 years ago
0
Feature/post dbt integration cleanup
#182
davhin
closed
2 years ago
0
Decrease user batches to solve out of memory problems.
#181
jakobkolb
closed
2 years ago
0
Remove docker layer caching during airflow CI unit tests.
#180
jakobkolb
closed
2 years ago
0
Next