-
Currently, the ETL pipeline is a loosely federated set of scripts, AWS Glue/Athena SQL queries, and calculated fields in QuickSight. Investigate AWS options for running arbitrary Python code. Future m…
-
### Windows Version
Microsoft Windows [Version 10.0.22631.4169]
### WSL Version
2.2.4.0
### Are you using WSL 1 or WSL 2?
- [X] WSL 2
- [ ] WSL 1
### Kernel Version
5.15.153.1-2
### Distro Ver…
-
I wonder if you could make suggestions on how to use this in an AWS glue job. My method does not involve using spark-submit but rather creating job definitions and run-job using boto3 tools.
W…
-
This is not a public issue, but one related to https://github.com/lewster32/corporallancot/milestone/1. We need to write an ETL script that transforms the old SQS IRC notes into a script that can be d…
-
@rtmill What do you think of doing this in the new Data Quality Dashboard? We just started using it and so far I am loving it.
https://github.com/OHDSI/DataQualityDashboard
-
I'm using Stitch for ETL and we recently onboarded with Kustomer.
The first week or so we were getting data through Stitch with this Kustomer integration just fine, but then it seemed that the extrac…
-
As discussed with Andrew Williams, it would be very helpful (also as a data quality metric) to be able to compute coverage of the unit tests.
One way to compute this would be to store the field-to-…
-
### Run Information
Name | Value
-- | --
Architecture | x64
OS | ubuntu 22.04
Queue | TigerUbuntu
Baseline | [6d838df6888da0060984526ea26960709447f304](https://github.com/dotnet/runtime/commit/6d8…
-
```
S3_BUCKET - s3 bucket of tags map file, default: skills-etl
S3_TAGS_MAP_KEY - s3 key of tags map file, default: tagsMap.txt
```
https://github.com/topcoder-platform/skills-etl
-
There are various circumstances where Hz simply gives up on the cluster. This could be things like OutOfMemory Exceptions or hitting the ulimit on sockets etc. A restart usually fixes the issue but we…