-
[INFO] 2020-09-24 17:17:54.245 - [taskAppId=TASK-6-21-23]:[121] - -> Connecting to jdbc:hive2://gdlt-b-master01.cnbdcu.com:2181,gdlt-b-master02.cnbdcu.com:2181,gdlt-b-master03.cnbdcu.com:2181,gdlt-b…
-
## Bug
### Describe the problem
When configure spark using [setup-configuration-s3-multi-cluster](https://docs.delta.io/latest/delta-storage.html#-setup-configuration-s3-multi-cluster
) In exampl…
-
在 byzer-engine-deployment 的 yaml 中,将插件加入到 driver 和 executor 的启动 path 中,部署启动后却没有加载到相关的类
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: byzer-engine
namespace: byzer
spec:
…
-
Are there any tips or support on setting up a Disaster Recovery (DR) environment with Apache Hudi?
We are creating our Datalake, stored on AWS S3, by running a Spark structured streaming applicatio…
-
## Senior Data Engineer - XP INC
Enquanto Engenheiro de Dados Sênior, você será responsável por criar processos que extraiam, transformem e carreguem os dados para uma central de dados (datalake), …
-
Hi,
In the field of compressor gzip is optimized for better compression ratio and is not recommended for streaming pipelines.
snappy is the most standard algorithm when doing streaming pipeline …
-
I try to create table and sysn to hive. However its show CreateHoodieTableCommand.orgsetter$nodePatterns_$eq(Lscala/collection/Seq;)V is abstract. I use hudi 0.11 + spark 3.2.1.
Environment Descri…
-
Follow below steps to find this problem
1. save delta table
```
set rawText='''
{"id":1,"content":"MLSQL是一个好的语言","label":0.0},
{"id":2,"content":"Spark是一个好的语言","label":1.0}
{"id":3,"content":"…
-
Enquanto Engenheirx de Dados Sênior, você será responsável por criar processos que extraiam, transformem e carreguem os dados para uma central de dados (datalake), atendendo as demandas do time de Ren…
-
Enquanto Engenheiro de Dados Sênior, você será responsável por criar processos que extraiam, transformem e carreguem os dados para uma central de dados (datalake), atendendo as demandas do time de Ren…