-
Notes 8/30:
* pipeline utils functions moved into each capsule (i.e. dont need special codebase to translate data between capsules)
* capsules are being added to github for each processing step (ex: a…
-
The test coverage ETL and the unittest ETL generate two different types of suite names. Ensure the two pipelines generate standard suites names that can be compared.
-
**Description**
There are many steps involved in adding a new data source to the ETL pipeline that populates the tool. It can be especially challenging to know which steps to take and to confirm that…
-
Includes
- OpenShift dev/test/prod ETL pipeline
- Ability to transform all data from postgres back to Oracle, and vice versa, must be bidrectional
- Ability to schedule jobs, or run it "live", depend…
-
HL7 to FHIR converter did not give any issues but running into this issue with Dicom. Using Same environment.
Exception in thread "main" java.lang.IllegalStateException: Unable to return a default …
-
Use python, airflow and Pyspark
-
The daily batch load to BigQuery dataset `public-data-finance.crypto_polygon` has seemed to stop running since 2024-09-01. Is there anyway we could resume this job?
-
I work at DBT and have been improving an ETL pipeline for gov.uk content we have based on parameters the department needs. I'd like to configure it so it ingests and overwrites data that's changed rat…
-
### Describe the bug
Since CodeCommit was deprecated (see [here](https://simonwillison.net/2024/Jul/30/aws-codecommit-quietly-deprecated/)), accounts/organizations that do not have existing CodeCommi…
-
Hello,
A nice and common thing is having a field (or column) description (sometimes called comment). It exists for database and several file formats. It's also pretty common in ETL/Dataprep tools s…