-
Maybe we should consider using [hypothesis](https://hypothesis.readthedocs.io/en/latest/) for testing. I use it here at work to test ETL and data pipelines and it works like a charm.
-
On running this pipeline: https://github.com/ploomber/projects/tree/master/templates/etl , got the following error:
```
PatoolError: patool can not unpack
patool error: error extracting ../ploomber…
-
## Context
The OT Metrics are computed on a VM. All the necessary files are copied to the VM and then the relevant metrics are generated.
We want to migrate the logic to accept `gs://` locations f…
-
This should be done as a pre-processing step, part of overall ETL pipeline, such that each individual analysis does not need to do normalization
Currently done for water packages here:
https://nbv…
-
### Describe the feature
Support KeepJobFlowAliveWhenNoSteps Or Auto-termination (after idle) in stepfunction creates EMR cluster.
### Use Case
Our team is using Stepfunction EMR (EmrCreateCluste…
-
有两个pipeline(单向,mysql5.7.16-log)跑着跑着突然就挂起,报了这个错,没找到其他相关日志。。
```
pid:15 nid:1 exception:setl:com.alibaba.otter.node.etl.select.exceptions.SelectException: java.lang.NullPointerException
at com.alib…
luyee updated
2 years ago
-
## 🚀 Feature
Recently, we implemented `bbs_database download` for multiple different sources. It might be a good idea to extend our integration test to actually use this download (rather than using…
-
BigQuery has pretty nice support for adding descriptions to tables and columns. We aren't yet making heavy use of it, but we _are_ starting to add some descriptions into JSON schemas in mozilla-pipeli…
-
Hello
I'm trying to replicate your example in my own project.
But I have an issue with python udf: always run into this error `ModuleNotFoundError: No module named 'pipelines'`
I simply changed …
-
Investigate whether it is possible to use SystemDS with CDAP.io or cloud data fusion instances.
https://cdap.atlassian.net/wiki/spaces/DOCS/overview
https://cloud.google.com/blog/products/data-ana…