-
Environment:
Spark version :3.2
Delta version :2.2.0
```
from pyspark.sql import SparkSession
import os
def init_spark():
spark = SparkSession.builder \
.appName("HiveMetasto…
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
I'm trying to read an excel file in databricks that has some very large text fields and I'm g…
-
I successfully installed spark-1.6.1-bin-hadoop2.4, CaffeOnSpark and mnist dataset, then I Adjusted ${CAFFE_ON_SPARK}/data/lenet_memory_train_test.prototxt to use absolute paths, such as.
"file:/hom…
-
**Used Spark version - 1.6.0**
**Used Spark Job Server version - 0.6.2**
**Deployed mode- default**
I am trying to create a spark job which would do a 'SELECT * ' on a Hive table and collect …
-
I'm creating this ticket as per @raver119 's request:
I'm training on a spark cluster with 4 slaves (all GPU machines with each 4 GPUs -- type Nvidia Tesla p40 with 24gb each).
I'm trying to fit a…
-
**Describe the problem you faced**
run spark-sql works
select * from bi_ods_real.ods_api_test_task_log_rt limit 10;
![image](https://github.com/apache/hudi/assets/15028279/0c624090-df36-43a6-ae…
-
## Bug
### Describe the problem
My simplified use case is to read from one location and append the data to a Delta Lake table with a Hive Metastore in batches. I have to do this for a couple…
-
For OAP-1.1, we run TPC-DS 99 queries for Power Test and we meeting hang issue when running q66.sql. The issue is reproduced on 2 clusters. Some messages are showed below:
org.apache.arrow.plasma.P…
-
Hello,
I'm using Spark 3.4.1, Scala 2.13.11, java 11 and import:
"com.gelerion.spark.sketches" % "spark-sketches" % "1.0.0"
I'm getting this error:
Exception in thread "main" java.lang.NoClassDe…
-
We have a high volume streaming service which works most of the time . But off late we have been observing that some of the parquet files written out by write flow are getting corrupted. This is manif…