-
https://github.com/saagie/gradle-saagie-dataops-plugin/wiki/projectsRunPipeline
Currently the output is
> Task :projectsRunPipeline
{"id":"c0ed1d49-8924-4d5b-bae9-59f536b84009"}
To be consist…
-
Support Ability to export all the artifacts of the project and import them into the target platform. This is not intended to create a new project but just the artifacts inside the project.
Refer t…
-
There are challenges in creating soil and weather files for large areas. This had been solved this by splitting the soil and weather file creation by Zones/provinces. There is now a bottle neck again,…
-
The current `dataops-ci-bot` is too noisy, and reports results on changes that are irrelevant to schema validation of the document sample. This could be reduced by checking the PR for changes to files…
-
**_THIS ISSUE DESCRIPTION IS STILL IN PROGRESS BECAUSE WE NEED LINK TO ALL CODE PARTIAL THAT NEED CHANGES:_**
In the project we have alot of side effect going on.
We need to reduce those side effe…
-
https://github.com/saagie/gradle-saagie-dataops-plugin/wiki/projectsStopJobInstance
The output currently is
saagies-MacBook-Pro:Gradle shivakumar$ gradle -b build.gradle.projectsStopJobInstance p…
-
Looks like EMAIL is still mandatory for this . I am not able to create a job if there is no email.
My Gradle file
plugins {
id "io.saagie.gradle-saagie-dataops-plugin" version "1.1.70"
}
…
-
Ingest new bundles for the HDCA 10x dataset in prod.
In the original submission, the bundling was incorrect. In order to fix this since the re-bundling is not possible at the moment, I would like t…
-
Hello,
I'm interested by proxySQL for query logging and ldap auth.
I installed it and configured backend server, users and query logging but i'm looking for documentation about ldap auth in prox…
-
先可以把所有的开源方案整合起来,后面如果发现不合适了再自己重写,这样原本的方案依然成为兼容其他平台的特性。
同时在手册中总结各种使用平台的数据工程管理经验,寻找打通这些平台的需求点,从而最大限度地拉低自研平台的难度。
- 数据集:使用OpenMetadata作为底层。
- 数据应用:使用App Engine作为底层。
- 数据处理:使用FaaS作为底层。
- 数据编排:使用AirFl…