-
### About the group
We want to make it easy for people who want to be active politically to find a nearby opportunity that matches their interests. Starting by scraping, normalizing and deduping e…
-
Hey there,
Thanks for the great library, the Go market really lacked it.
Currently I am working on a data processing pipeline which imports data from various sources. This means that I have a da…
-
The current `candidates_to_merge` implementation results in several silent but significant bugs, including **data loss**.
For example, let's say the `time_threshold` is 3 minutes and we have the f…
-
### Feature description
When a user tries to force stop a pipeline, the result is that the status of the pipeline ends up being `degraded` and not `stopped`.
I think it would make sense that it…
-
## Describe the task
The objective is to integrate Great Expectations into our Python ETL pipeline to ensure data quality. The task involves researching various integration methods, documenting the…
-
Come up with a strategy to storage changing data from multiple days.
-
Hi,
I'm using drop 1.3.3 in order to do aberrant expression analysis. However during outrider execution with 2 bam data and 9 external counts, I'm getting the following error:
```
[Tue May 30 16:…
-
**Is your feature request related to a problem? Please describe.**
Currently our ETL job runs every 30 minutes and inserts a file into S3, triggering OpenSearch ingestion pipeline. Due to varying ETL…
-
**Is your feature request related to a problem? Please describe.**
I have a data pipeline built as a combination of AOSS pipeline and AOSS collection. This pipeline is a real time monitor for logs.
…
-
Explore & implement methods to get more details from FB and Google APIs, since it's proving difficult to do a "blanket search" for all businesses in RIVCO/cities using those APIs.
1. Yelp is good f…