nestauk daps_utils issues

nestauk / daps_utils

Tools for setting up and running pipelines in a Data Analytics and Production System (DAPS).

0 stars 1 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Refactor `talk_to_luigi` into `DapsFlowMixin`, and integrate into batch steps

#120 jaklinger closed 1 year ago
0
Investigate removing `talk_to_luigi`

#119 jaklinger closed 1 year ago
0
[117] Docker prune

#118 jaklinger closed 2 years ago
0
Implement docker prune before build

#117 jaklinger closed 2 years ago
0
Add mysql reqs from ojd-daps

#116 jaklinger closed 2 years ago
0
Add mysql reqs to `requirements.txt`

#115 jaklinger closed 2 years ago
0
[113] Updated sqlachemy-utils version

#114 jaklinger closed 2 years ago
0
Upgrade `sqlalchemy-utils`

#113 jaklinger closed 2 years ago
0
[111] detect project name from pyproject.toml

#112 jooh closed 2 years ago
2
`calver-init`: add support for projects that diverge from `project_name/project_name` directory structure

#111 jooh opened 2 years ago
0
Improve legibility / maintainability of `docker_utils.format_logs`

#110 jaklinger opened 3 years ago
0
[107] apply `black` formatting

#109 jaklinger closed 3 years ago
0
[107] curate stability

#108 jaklinger closed 2 years ago
0
Improve curate stability

#107 jaklinger closed 2 years ago
0
[105] robust connections with `pool_pre_ping`

#106 jooh closed 3 years ago
0
More robust DB connections with `pool_pre_ping`

#105 jooh closed 3 years ago
0
Move db functionality into flow mixin

#104 jaklinger closed 3 years ago
1
Allow batch to access DB

#103 jaklinger closed 2 years ago
0
`stale_breadcrumb` feature

#102 jaklinger closed 3 years ago
1
Allow running of curate on previously run flows

#101 jaklinger closed 3 years ago
0
Add the __version__ and git.tag to AWS batch environment variables

#100 jaklinger closed 2 years ago
0
Change order of text processing in log manipulation

#99 jaklinger closed 3 years ago
1
Unicode decode error in logs

#98 jaklinger closed 3 years ago
0
[96] Fix truncation error

#97 jaklinger closed 3 years ago
0
Logs are truncated by character, not lines

#96 jaklinger closed 3 years ago
0
Set up a team schedule and guidelines for production failures

#95 jaklinger opened 3 years ago
0
Holistic evaluation of dependency management

#94 jaklinger opened 3 years ago
0
[92] Truncated logs

#93 jaklinger closed 3 years ago
1
Production error messages are too long

#92 jaklinger closed 3 years ago
0
[79] add merge driver, refactor

#91 jooh closed 3 years ago
4
Data Engineering Cookiecutter

#90 jaklinger opened 3 years ago
0
Set up ontology baking

#89 jaklinger opened 3 years ago
0
Bake data dictionary into the ORM

#88 jaklinger opened 3 years ago
0
Add `declarative_base` (as in `ojd_daps`) to `daps_utils.__init__`

#87 jaklinger opened 3 years ago
0
Refactor all of `dev_deploy` to daps_utils as a standalone executable

#86 jaklinger opened 3 years ago
0
Add all of `ojd_daps.deploy` as a templates to `daps_utils`

#85 jaklinger opened 3 years ago
0
[77] Add support for instrumented descriptors

#84 jooh closed 3 years ago
0
Refactor vanilla concrete implementations of `CurateTask` such as `EnrichmentCurateTask`

#83 jaklinger opened 3 years ago
0
Refactor `PrepareDatabaseTask` to `daps_utils`

#82 jaklinger opened 3 years ago
0
Add tests to ensure that all required GH secrets are in place (see `ojd_daps`)

#81 jaklinger opened 3 years ago
0
Copy `ojd_daps` actions as templates

#80 jaklinger opened 3 years ago
1
Action to resolve `VERSION` merge conflicts on all active PRs

#79 jaklinger closed 3 years ago
7
Update python version in setup.py

#78 jaklinger closed 3 years ago
0
object_as_dict doesn't work with ORM-instrumented descriptor queries

#77 jooh closed 3 years ago
1
[NestaBot] Add a License for this repo

#76 jaklinger opened 3 years ago
0
Fix failed AWS access when Luigi tasks running as Batch jobs

#75 dtprogammer closed 3 years ago
0
Swap back to the metaflow security group

#74 jaklinger opened 3 years ago
3
Separation of metaflow resources across DAPses

#73 bishax opened 3 years ago
0
CI/CD for metaflow cloudformation templates

#72 bishax opened 3 years ago
3
[70] Dynamic retrieval and setting of aws env vars

#71 jaklinger closed 3 years ago
0