-
I know it sounds weird to support Databricks because you can execute spark code directly on databricks. But it would be Nice to be able to use a Databricks SQL Warehouse with pyspark code.
-
### Product
Amazon Linux 2023
### What is your question?
building archive for EMR serverless takes a lot of time when using al2023
Dockerfile
```
FROM amazonlinux:2023 AS base
RUN yum insta…
-
This is unrelated to #671 because I don't see any log of the following type in the spark operator logs.
```
"Executor pod xxx not found, assuming it was deleted.
```
#### VERSIONS
Operator = …
-
-
### Name and Version
bitnami/spark:3.4.0
### What architecture are you using?
None
### What steps will reproduce the bug?
`pyspark` command is supposed to open the pyspark shell, but it keeps ret…
-
### Description
pretty much all Python calculation engines support show method , see pyspark documentation https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFr…
-
Is there a pyspark implementation of this module. Thanks
-
would it be possible to use nbdime/ndbiff inside PySpark Notebooks? Often these type of notebooks have a specialized jupyter notebook format, e.g. Azure Synapse Analytics Notebooks.
-
## Describe the bug
When running on Databricks in Spark Connect mode (for example Shared Isolation mode, Job Cluster, DBR 15.4) - Spark Connect isn't detected by koheesio, which leads to exceptions i…
-
## Description of issue
when using the `pandera.pyspark` module, creation of new DataFrameSchema instance throws a `TypeError` when the `unique` parameter is included in `Column` class initialisati…