-
**Is your feature request related to a problem? Please describe.**
Filing this to investigate further. Spark added new node EmptyRelationExec.
pr: https://github.com/apache/spark/pull/46830
issue…
-
**Describe the problem you faced**
I have one job insert into a new partition, job attemp1 failed due to shuffle fetch failed (internal environment problem).
I rerun this job (job attemp2), bu…
-
### Is your feature request related to a problem? Please describe
## 1. Current status
Currently, users can use the Spark Dataset API to directly read and write OpenSearch indices. The OpenSearch …
-
### What is the problem the feature request solves?
I'm running the 1TB TPCDS benchmark over Comet and Vanilla Spark.
I'm running on a 48Core 186G RAM machine
Here's my config:
```
/localhdd/…
-
### Willingness to contribute
Yes. I can contribute a fix for this bug independently.
### OpenHouse version
v0.5.62
### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 20.…
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Describe the bug
Writing data with sparse vector gets unrecognized class error:
```
fields = [
Field…
-
When translating sql on a `Spark SQL` connection, the 'add_year' translation is double quoting the column name:
```r
# SQL: add_years(dob, 1)
ADD_MONTHS('`dob`', 1.0 * 12.0)
```
This fails on…
-
HUDI version -> 0.14.1
Spark version -> 3.2.0
hadoop version -> 3.1.1
hive version -> 3.1.1
Hi
I wanted to use partial data update payload. I have multiple sources, which all want to write into…
-
I use the lastest nightly version 1.7.0b20240501.dev0.
It works init spark and read data to spark dataframe, but when run `ray.data.from_spark(df)` :
it blocked when using spark 3.4.3.
and when…
-
### Apache Iceberg version
1.5.0 (latest release)
### Query engine
Spark
### Please describe the bug 🐞
Previously my pipeline was using iceberg 1.3 on Dataproc (image version 2.1 which has spark …