-
I have two Spark notebooks running in VS Code, latest extension version, Spark runtime 1-2. The notebooks mostly run fine, but on occasion, for example if accidentally closing VS Code while runtime co…
-
I tested Spark Notebook with the following simple code:
``` scala
val someValues = sc.parallelize(1L to 10000L)
case class Foo(key: Long, value: Long)
val someDummyGrouping = someValues.map(v => (v /…
-
**Is your feature request related to a problem? Please describe.**
Currently, TiSpark doesn't support a MySQL/TiDB partition table syntax `select col_name from table_name partition(partition_name)`
…
-
```
spark.sql("""CREATE TABLE iceberg.glue_catalog.iceberg_table ( order_id bigint,
customer_id bigint, order_amount DECIMAL(10,2), category string, trans_ts Timestamp)
USING iceberg
location 's3:…
-
When we disable spark.oap.sql.columnar.preferColumnar and set spark.oap.sql.columnar.joinOptimizationLevel equal to 6. The query plan of continuous joining of q64.sql cannot successfully go back to va…
-
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](http…
-
**Description**
Antlr class conflict when running SQL code in a notebook using Delta Lake.
**Reproduction**
Steps to reproduce the behavior:
1. ...
2. ...
**Environment**
Using a varian…
-
### Describe the bug
I have a simple model (for the sake of argument called my_table) which loads data from an existing iceberg table (not created with dbt-glue) and writes to a new iceberg table:
`…
-
### SynapseML version
0.11.2
### System information
- **Language version** python 3.9
- **Spark Version** 3.4.1
- **Spark Platform** AWS EMR
### Describe the problem
LightGBM consi…
-
I used pyrasterframes v0.10.1 with Databricks 9.1 LTS ML (includes Apache Spark 3.1.2, Scala 2.12).
I get the following error and cannot access the dataframe.
```
-------------------------------…