-
## Is your feature request related to a problem? Please describe.
Pydantic enforces strict types. In the current implementation all Spark related logic (readers, writers, transforms, integr…
-
The docs say this isn't supported but putting this out there as an enhancement request. Definitely not easy, can't find anything outside of SAS that writes to SAS7BDAT files.
-
Gaffer has a Spark library with Scala and Java APIs for accessing data using Spark; generating RDDs and Spark DataFrames from Gaffer graphs.
Gaffer also has a python shell with implementations of s…
-
Currently, `embedchain` allows the addition of various types of data sources such as YouTube videos, PDF files, and web pages to be processed and used in the application. This feature request proposes…
-
> This repo contains a library for loading and storing TensorFlow records with Apache Spark. The library implements data import from the standard TensorFlow record format ([TFRecords] (https://www.ten…
-
TL;DR
I ran across a bug in spark 3.5 and graphframes 0.8.3 where when using connected components, if `spark.sql.adaptive.enabled` is not `false`, the results returned will be incorrect, with many ed…
-
### Apache Iceberg version
1.5.2 (latest release)
### Query engine
Spark
### Please describe the bug 🐞
Hello,
We have an existing working Spark Scala job (Spark 3.2.0 Iceberg 1.4.0 Sca…
-
Based on the discussion [here](https://gitter.im/polynote/polynote?at=5e29f22344c076313cd3611b)
Polynote has a great [example notebook](https://github.com/polynote/polynote/blob/master/docs/example…
-
**Describe the bug**
When using %%send-to-spark with a local pandas df which contains '\n' or ' (single quote) character, the command fails with the following error
`
SyntaxError: invalid syntax
…
-
If it isn't possible already, it would be nice i we can use spark dataframes to write to glue tables using something similar to wranglers to_parquet method. It works great for pandas and has the abili…