-
This is to create an issue for tracking a more organized ETL pipeline. This can be a script or a notebook, but as long as we organize it so that we have the ability to update all sources at once in a …
-
### Describe the feature
In certain contexts an `ExEx` requires access to comprehensive data from all pipeline stages including hashed state, merkle, transaction lookup and history. Currently the pip…
-
### Motivation
The ETL needs to be able to fetch 5 mins of data, every 5 minutes, in such a way that each fetch updates historical records allowing us to understand when previous predictions became…
-
## 🐛 Problem
Hi, we are trying to use the FinOps Hub & its PowerBI reporting as a POC. As we don't have access to EA or MCA costings, we are using the [sample FOCUS export report](https://microsoft.…
-
We're looking to enhance Elsa's data processing capabilities. To do this, we're drawing ideas from Azure Data Factory. Here's a breakdown of the new concepts we're considering for integration:
1. *…
-
### Discussed in https://github.com/apache/airflow/discussions/33556
Originally posted by **ntnhaatj** August 20, 2023
### Description
Hi, as [my issue was raised here](https://github.com/a…
-
i tried run the "Streaming ETL pipelines in Python with Airbyte and Pathway"
and for many sources and i kept getting the folllowing error :
Traceback (most recent call last):
File "/home/hisha…
-
## User Story
In order to harvest WAF sources effectively and at scale, datagovteam would like to harden the current WAF ETL pipeline.
## Acceptance Criteria
[ACs should be clearly demoable/v…
-
### What product(s) is this story for?
OpenChallenges
### As a user, I want
_No response_
### Description
The goal of this story is to update the existing `openchallenges-edam-etl` to effectively…
-
Hi,
i have 10 ETL pipelines which all run more or less at the same time, each pipeline runs based on the same input data object with a unique id. I would like to have all my etl pipelines contribut…