prefeitura-rio / pipelines_rj_smtr

Códigos de captura e tratamento de dados da SMTR

https://pipelines.dados.rio/

1 stars 0 forks source link

[WIP] Refatoração Pipelines #8

Closed pixuimpou closed 4 months ago

pixuimpou commented 5 months ago

ChangeLog

Cria novo flow de captura generico
Nova estrutura de pastas do projeto
Migração pipelines de captura de bilhetagem

mergify[bot] commented 5 months ago

@pixuimpou esse pull request tem conflitos 😩

github-actions[bot] commented 4 months ago

Análise da árvore de código

Os seguintes arquivos são afetados diretamente por alterações realizadas nesse pull request:

pipelines/utils/prefect.py
pipelines/utils/incremental_capture_strategy.py
pipelines/templates/__init__.py
pipelines/templates/backup/__init__.py
pipelines/treatment/__init__.py
pipelines/flows.py
pipelines/treatment/templates/__init__.py
pipelines/utils/backup/__init__.py
pipelines/capture/templates/tasks.py
pipelines/tasks.py
pipelines/treatment/bilhetagem/__init__.py
pipelines/treatment/templates/tasks.py
pipelines/exemplo/nome_do_objetivo/flows.py
pipelines/capture/jae/tasks.py
pipelines/utils/capture/gcs.py
pipelines/utils/fs.py
pipelines/utils/pretreatment.py
pipelines/br_rj_riodejaneiro_brt_gps/constants.py
pipelines/constants.py
pipelines/schedules.py
pipelines/treatment/templates/flows.py
pipelines/utils/backup/utils.py
pipelines/utils/capture/api.py
pipelines/capture/jae/flows.py
pipelines/treatment/bilhetagem/flows.py
pipelines/br_rj_riodejaneiro_brt_gps/tasks.py
pipelines/utils/backup/tasks.py
pipelines/capture/jae/__init__.py
pipelines/utils/jinja.py
pipelines/capture/jae/constants.py
pipelines/utils/dbt_vars.py
pipelines/capture/__init__.py
pipelines/utils/gcp.py
pipelines/utils/capture/__init__.py
pipelines/utils/utils.py
pipelines/capture/templates/flows.py
pipelines/br_rj_riodejaneiro_brt_gps/flows.py
pipelines/utils/secret.py
pipelines/templates/backup/tasks.py
pipelines/capture/templates/__init__.py
pipelines/utils/capture/base.py
pipelines/utils/capture/db.py