-
I'm at a loss of how to do some basic math and logic in SparklyR (v1.7.5, R 4.1). My actual table is ~60M rows x 600 columns, with values like the ones below.
Here's a basic example:
> test …
-
**Describe the problem you faced**
I am facing issues for upsert operation in hudi 0.14 RLI in EMR 6.15 Spark 3.4.1 using "Record level Index".
i see insert mode working as expected but upsert ope…
-
Hi All,
I am new to Spark and Scala. I have the source code for Spark SQL Performance Tests and dsdgen .
Can anyone tell me how to proceed next ? I am done with building by giving command bin/run…
-
## Background
Conformance Rules can be difficult to understand.
## Feature
We can add tooltips to the rules which provide a concise explanation.
## Proposed Solution [Optional]
Conformance …
-
Here is my code:
```
select
owner,
owner_email,
owner_mgr,
owner_mgr_email,
week_begin,
actual_hour,
working_days*8 as working_hour
from(
select
o…
-
# spark 2.0 踩过的SparkSession的坑
取代了SQLContext(HiveContext)的SparkSession
## 背景
我的服务端的逻辑是在actor内部进行的,但发现多个actor中执行的过程中,访问到了其他actor内部session中注册的临时表
## 抽象的运行代码
actor的逻辑大概可以抽象成这样:
```s…
-
Hi, I created a PySpark function that writes into TileDB like this:
```python
sample_idx = sample_annotation.sample_idx.values
geno_path = OUTPUT_PATH
def spark_ingest_vcf(batch_iter):
for …
Hoeze updated
3 years ago
-
**Is your feature request related to a problem? Please describe.**
For Spark we are pushing to get more support for structs in a number of operators. We already have some support for sorting structs…
-
In our current migration effort to spark-connector new version, we noticed there are some type conversion issues after using spark-mssql-connector ver 1.0.2 for BulkCopy. I have created two simple un…
-
## Problem Description
This design proposal is for adding feature request #229.
Currently, Hyperspace supports creating indexes only on data with fixed schema. This means:
- All columns from "…