-
Hi. Are there any plans about when to add SparkSQL as a source?
-
Usages of the SparkSQL function [`def struct(cols: Column*): Column`](https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.struct.html) fail with an error me…
-
**Describe the bug**
when processing SQL scripts where the grammar is backwards, `LineageRunner().target_tables` fails to parse/fix.
**SQL**
Paste the SQL text here. For example:
```sql
FROM `d…
-
**Is your feature request related to a problem? Please describe.**
Provides users with the ability to connect and set up `Spark SQL` as a data source, enabling integration with distributed data proce…
-
## 背景
目前的体系中,SparkSQL主要提供给ad-hoc类的OLAP查询。SparkSQL通过metastore获取hive表信息,因而可以直接查询Hive表的数据。metastore的性能直接影响SparkSQL的查询速度。
这次的问题是从用户上报的一个case(属于第一类问题)开始追查,过程中出现了很多和“想象”的场景不一样的情况。最终的结果可能很简单,过程中使用到的工具值…
-
@krassowski we have developed a jupyterlab extension which provides code completion from Spark sql and Trino.
You can see the features of the extension here https://github.com/CybercentreCanada/jup…
-
### Description
I'm trying to run TPC-H Q3 and compare the performance between Wayang and SparkSQL under the following setup:
* Running both Spark (3.5.1) and Wayang on a local VM with 32 CPU co…
-
### Description
The context of this issue is re-introducing SparkSQL regexp_replace. The current PR to do this is #8333. This issue was discovered after @mbasmanova 's comment on that PR "_Would yo…
-
Hey there, thanks for putting this together.
As another data point, I've written a SparkSQL implementation of this challenge - see https://github.com/SamWheating/1trc. I've included all of the ste…
-
### Description
Gluten implemented some logic to convert call expr as subfield filter. To avoid duplication, we would like to use the existing 'leafCallToSubfieldFilter' logic in Velox. One incompati…