-
would it be possible to use nbdime/ndbiff inside PySpark Notebooks? Often these type of notebooks have a specialized jupyter notebook format, e.g. Azure Synapse Analytics Notebooks.
-
jupyter/pyspark-notebook:latest
-
spark:python3-java17
bitnami/spark:3.5.2
quay.io/jupyter/pyspark-notebook:latest
-
Hi đź‘‹
We are currently experimenting with using `sparkdantic` on our Spark schema definitions in our pipelines inside Databricks. However, based on our current configuration, we are bound to install…
-
## Description
The PySpark profiling example notebook uses the `collect_column_profile_views` method as the first example which returns a dictionary and is not easy to then use to write to WhyLabs.…
-
### What docker image(s) is this feature applicable to?
base-notebook, docker-stacks-foundation, minimal-notebook
### What change(s) are you proposing?
Hi,
I would like to propose the addition …
-
I get the following exception in "df.groupby('partition_id').count().show()" cell when running the Loading Data -notebook:
```
java.lang.OutOfMemoryError: Java heap space
at org.apache.spark.sql.…
-
### SynapseML version
0.10.1
### System information
Language version: Python: 3.8.10, Scala 2.12
Spark Version : Apache Spark 3.2.1,
Spark Platform: Databricks
### Describe the problem…
-
Found this on Zeppelin users list as shared by moon.
Great idea!
Does this support submitting pyspark notebooks too?
Thanks.
Tagar updated
6 years ago
-
### System Information
- **Spark or PySpark**: PySpark
- **SDK Version**: v1.2.8
- **Spark Version**: v2.3.2
- **Algorithm (e.g. KMeans)**: n/a
### Describe the problem
I'm following the instr…