-
I use metorikku reading data from hive, then write to hdfs with parquet format. But the out is always empty. I can not figure out what is wrong, can someone give me some advice? Thanks.
The job conf…
-
### Is there an existing issue for this?
- [X] #14388
### Who can help?
@maziyarpanahi
### What are you working on?
Been struggling with running a simple demo of the SparkNLP Sentiment Analysis,…
-
I'm running Spark job written in F# in GCP DataProc, and everything works great.
By default, GCP does not gather logs from the Job Output and I can find logs only in GCP console, which is not accep…
-
Leading to the following exception in any job where Spark attempts to compress data:
```
ERROR GPLNativeCodeLoader: Could not load native gpl library
java.lang.UnsatisfiedLinkError: no gplcompressi…
-
## Problem
### How much memory does a spark-dependencies job take while handling about 12Gb data index?
I am totally new to the spark project and I have tried serval times to run a spark-depend…
-
**Describe the bug**
Here it refers to installing spark and a worker process:
https://github.com/dotnet/spark/blob/main/docs/getting-started/windows-instructions.md
However when you download both…
-
1) 이미지 데이터 넣었을 때
임베딩 그대로 넣었을 때랑, 이미지 유사도 만 넣었을 때...
2) 텍스트 데이터 넣었을 때
=> 피처 코사인 유사도(평균, 중앙값, 1분위수, 2분위수, 3분위수 첨도, 왜도, 표준편차...), 임베딩(평균, 중앙값, 1분위수, 2분위수, 3분위수 첨도, 왜도, 표준편차...) 데이터 그대로 넣었을 때....
3) 이…
-
Is Google Cloud support envisioned for the future?
Some things that I'd like to see:
- Google Cloud Storage as a repo, analogous to S3
- DataProc as a notebook runner, analogous to EMR
- Ability…
-
Despite everything I have tried, I still cannot figure out how to configure logging for my Spark application. The `log4j.properties` file is always ignored.
### log4j.properties in the resources
…
-
Given the proliferation of arm based cloud instances and the nvidia arm platforms it would make sense to support running on arm64.