-
I was watching the ETL job run in prod and noticed that a few requests (about 7 out of 2,500) to our back-end failed due to timeout.
I could not see a pattern as to which dshop or type of file would …
-
## Feature request
**Is your feature request related to a problem? Please describe.**
We are using starrocks-spark-connector to get computation result from starrocks, and we have a lot of sp…
-
Blocked by #47
-
@JanssenBrm: web app and job_tracker should log to https://etl-dev.terrascope.be instead.
![openeo-dev_to_etl_prod](https://github.com/Open-EO/openeo-geopyspark-driver/assets/1032518/f6031c72-6d56-…
-
I'm trying to index GR18 data and encountering an error when I run. It is expecting `chainId` to come as an int not a string.
```bash
yarn run etl --chain [chainId]
```
I also tried running th…
-
Choose orchrestration tool to implement batch jobs. Will most likely be Airflow, but could also consider other options like Prefect or Databricks.
Use dbt to implement Batch ETL jobs, including pre-a…
-
### Definition
Set up a data transformation process and tool that transforms the data from the current grants.gov live data model into the new data model for simpler.grants.gov.
### Business Goals
-…
-
### Is your feature request related to a problem? Please describe.
No. It's a common senario, we use flink as a streaming processor to do ETL jobs saving data to Hive.
### Describe the solution you'…
-
I wonder if you could make suggestions on how to use this in an AWS glue job. My method does not involve using spark-submit but rather creating job definitions and run-job using boto3 tools.
W…
-
#### Setup:
ArangoGraph Oasis 3.11 (oneshard model, 3 x 4GB)
AWS Glue 4.0 - Spark 3.3, Scala 2, Python 3
ArangoDB Spark Connector [version 1.7.0](https://mvnrepository.com/artifact/co…